Maintenance-free Controllers

Deep Learning for Process Control

Traditional controllers used in the process industries, such as PID loops or MPC, require constant attention and upkeep for its entire lifecycle including modeling, design, tuning and maintenance. These controllers are typically designed for a narrow operating range by assuming linear process behavior, and are not resilient to changes in plant equipment or operating conditions. In complex industrial systems, there often exists a trade-off between high performance controllers and the development of complex models that are computationally intensive and difficult to interpret or maintain.

Photo of industrial plant and piping

Inspired by the recent successes of deep learning in computer vision and natural language processing, our group is exploring deep reinforcement learning (DRL) as a model-free and maintenance-free framework for process control in industrial settings. Recent work that we’ve published shows promising results for DRL in terms of setpoint tracking performance and adaptability, but there are still many fundamental questions left to explore, including sample efficiency (big data is not always good data), stability guarantees, interpretability and computational challenges.

Ultimately, we are interested in the development of smart plants and advanced controllers that can provide a high level of safety and reliability for the industry.

Reinforcement Learning based Design of Linear Fixed Structure Controllers

Reinforcement Learning based Design of Linear Fixed Structure Controllers
Reinforcement Learning based Design of Linear Fixed Structure Controllers - Lawrence et al. (2020): A standard closed-loop structure is shown inside the dashed box. Arrows passing the dashed line indicate the passing of some time-horizon [0, T]. Outside the dashed box, we store cumulative rewards based on slightly perturbed policies, which are used to update the policy with a finite-difference scheme described in section 4.2 in the manuscript.

Optimal PID and Antiwindup Control Design as a Reinforcement Learning Problem

Optimal PID and Antiwindup Control Design as a Reinforcement Learning Problem
Optimal PID and Antiwindup Control Design as a Reinforcement Learning Problem - Lawrence et al. (2020): The actor (PID controller) on the left is simply linear combination of the state and the PID & antiwindup parameters followed by a nonlinear saturation function. The critic on the right is a deep neural network approximation of the Q-function whose inputs are the state-action pair generated by the actor.

Selected Publications

See below for a selection of our papers related to deep (reinforcement) learning.

  1. Conference Proceedings NeurIPS Spotlight
    Almost Surely Stable Deep Dynamics
    , , ,
    In Proceedings of Advances in Neural Information Processing Systems 33 (NeurIPS 2020). 2020 [PDF] [Code] [Poster] [Video] [arXiv]
  2. Conference Proceedings
    Deep Neural Network Approximation of Nonlinear Model Predictive Control
    ,
    In proceedings of IFAC World Congress (To Appear). 2020 [PDF] [Slides] [Video]
  3. Conference Proceedings
    Optimal PID and Antiwindup Control Design as a Reinforcement Learning Problem
    , , , , ,
    In proceedings of IFAC World Congress (To Appear). 2020 [PDF] [Video] [arXiv]
  4. Conference Proceedings
    Reinforcement Learning based Design of Linear Fixed Structure Controllers
    , , , , ,
    In proceedings of IFAC World Congress (To Appear). 2020 [PDF] [Video] [arXiv]
  5. Journal Paper Top 10% Most Downloaded
    Towards Self-Driving Processes: A Deep Reinforcement Learning Approach to Control
    , , , ,
    AIChE Journal. 2019 [PDF]