Stability-constrained Learning: A Lyapunov Approach

Learning-based methods have the potential to solve difficult problems in control and have received significant attention from both the machine learning and control communities. Despite the good performance during training, the key challenge is that standard learning techniques only consider a finite number of system trajectories, and good performance during training does not give guarantees on how a learned controller would perform on new situations and real systems. Particularly, a neural network controller with a low training loss may actually lead to system instabilities (i.e., unbounded states and oscillations) when implemented during testing. For safety-critical systems from electric power grids to aircrafts, vehicles, and robots, the possibility of instability--created by the controllers themselves--is a deal-breaker. System operators would rather use the tried-and-true linear controllers than implementing learning-based control laws without guarantees.

The central goal of our project is to unlock the full capability of learning-based control for real-world engineering systems by providing formal stability guarantees. The unifying principle we adopt is to push control theory as far as possible for an application, then to use learning. Classical control methods are generally not powerful enough to design or synthesis good controllers, but they can provide "structural insights" on what a good controller should have. On the other hand, RL with neural networks is very good at searching over large parametric spaces, but they typically lack structure. Our research in this direction provides a bridge between classical control and learning, where theory fixes the "right" structure, and ML/RL optimizes the controllers that have this structure. 

Certifiably Robust Forward Invariance in Neural ODEs (Control for Learning)

Stability-constrained Reinforcement Learning (Learning for Control)

Communication-embedded Controller for Centralized, Distributed, and Decentralized Learning in Multi-agent Systems

References:

[1] Yujia Huang, Ivan Dario Jimenez Rodriguez, Huan Zhang, Yuanyuan Shi, and Yisong Yue, "FI-ODE: Certifiably Robust Forward Invariance in Neural ODEs", Arxiv preprints, 2024.

[2] Wenqi Cui, Yan Jiang, Baosen Zhang, and Yuanyuan Shi, "Structured Neural-PI Control for Networked Systems: Stability and Steady-State Optimality Guarantees", in Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023. 

[3] Jie Feng, Yuanyuan Shi, Guannan Qu, Steven Low, Anima Anandkumar, and Adam Wierman, “Stability Constrained Reinforcement Learning for Real-Time Voltage Control in Distribution Systems”, IEEE Transactions on Control of Network Systems, 2023.

 [4] Jie Feng, Wenqi Cui, Jorge Cortes, and Yuanyuan Shi, “Bridging Transient and Steady-State Performance in Voltage Control: A Reinforcement Learning Approach with Safe Gradient Flow”, IEEE Control Systems Letters, 2023.

[5] Sahin Lale, Yuanyuan Shi, Guannan Qu, Kamyar Azizzadenesheli, Adam Wierman, and Anima Anandkumar, "KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systems", IEEE Conference on Decision and Control (CDC), 2023.

[6] Yuanyuan Shi, Guannan Qu, Steven Low, Anima Anandkumar, and Adam Wierman, “Stability Constrained Reinforcement Learning for Real-Time Voltage Control", in American Control Conference (ACC), 2022.

Sponsers