PAPER (2021)

Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning

Control Engineering Practice,

, , , , ,

[PDF]

Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning by Nathan P. Lawrence, Michael G. Forbes, Philip D. Loewen, Daniel G. McClement, Johan U. Backström, R. Bhushan Gopaluni
Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning by Nathan P. Lawrence, Michael G. Forbes, Philip D. Loewen, Daniel G. McClement, Johan U. Backström, R. Bhushan Gopaluni

Click to enlarge image.

Abstract

Deep reinforcement learning (RL) is an optimization-driven framework for producing control strategies for general dynamical systems without explicit reliance on process models. Good results have been reported in simulation. Here we demonstrate the challenges in implementing a state of the art deep RL algorithm on a real physical system. Aspects include the interplay between software and existing hardware; experiment design and sample efficiency; training subject to input constraints; and interpretability of the algorithm and control law. At the core of our approach is the use of a PID controller as the trainable RL policy. In addition to its simplicity, this approach has several appealing features: No additional hardware needs to be added to the control system, since a PID controller can easily be implemented through a standard programmable logic controller; the control law can easily be initialized in a 'safe' region of the parameter space; and the final product—a well-tuned PID controller—has a form that practitioners can reason about and deploy with confidence.

Read or Download: PDF

Can't find a paper? Create a GitHub issue to request a preprint.


DAIS Lab Publications

Read More: