Reinforcement Learning Policies With Local LQR Guarantees For Nonlinear Discrete-Time Systems

Samuele Zoboli (1, 2) , Vincent Andrieu (1) , Daniele Astolfi (1) , Giacomo Casadei (3) , Jilles S Dibangoye (2) , Madiha Nadri (1)

1 LAGEPP - Laboratoire d'automatique, de génie des procédés et de génie pharmaceutique
2 CHROMA - Robots coopératifs et adaptés à la présence humaine en environnements dynamiques
3 AIS - Ampère, Département Automatique pour l'Ingénierie des Systèmes

Optimal control of nonlinear systems is a difficult problem which has been addressed by both the Control Theory (CT) and Reinforcement Learning (RL) communities. Frequently, the former relies on the linearization of the system thus obtaining only local guarantees. The latter relies on data to build model-free controllers, focused solely on performances. In this paper we propose a methodology to combine the advantages of both approaches, casting the formulation of an optimal local Linear Quadratic Regulator (LQR) into a Deep RL problem. Our solution builds on the linear framework to derive a learnt nonlinear controller showing local stability properties and global performances.

Domaines: Systèmes et contrôle [cs.SY]; Informatique [cs]; Intelligence artificielle [cs.AI]

Samuele Zoboli, Vincent Andrieu, Daniele Astolfi, Giacomo Casadei, Jilles S Dibangoye, et al.. Reinforcement Learning Policies With Local LQR Guarantees For Nonlinear Discrete-Time Systems. CDC, Dec 2021, Texas, United States. ⟨10.1109/CDC45484.2021.9683721⟩. ⟨hal-03353584⟩