Stability-constrained Learning: A Lyapunov Approach

Learning-based methods have the potential to solve difficult problems in control and have received significant attention from both the machine learning and control communities. Despite the good performance during training, the key challenge is that standard learning techniques only consider a finite number of system trajectories, and good performance during training does not give guarantees on how a learned controller would perform on new situations and real systems. Particularly, a neural network controller with a low training loss may actually lead to system instabilities (i.e., unbounded states and oscillations) when implemented during testing. For safety-critical systems from electric power grids to aircrafts, vehicles, and robots, the possibility of instability--created by the controllers themselves--is a deal-breaker. System operators would rather use the tried-and-true linear controllers than implementing learning-based control laws without guarantees.

The central goal of our project is to unlock the full capability of learning-based control for real-world engineering systems by providing formal stability guarantees. The unifying principle we adopt is to push control theory as far as possible for an application, then to use learning. Classical control methods are generally not powerful enough to design or synthesis good controllers, but they can provide "structural insights" on what a good controller should have. On the other hand, RL with neural networks is very good at searching over large parametric spaces, but they typically lack structure. Our research in this direction provides a bridge between classical control and learning, where theory fixes the "right" structure, and ML/RL optimizes the controllers that have this structure. 

Certifiably Robust Forward Invariance in Neural ODEs (Control for Learning)

Figure: Overview of our FI-ODE framework. We first pick a Lyapunov function based on the shape of the forward invariant set: the boundaries of the Lyapunov sublevel sets are parallel to the boundary of the forward invariant set. Then we show that robust forward invariance implies robust control and classification. We train the dynamics to satisfy robust FI conditions via robust Lyapunov training. To certify the forward invariance property, we sample points on the boundary of the forward invariant set and verify conditions hold everywhere on the boundary.

Stability-constrained Reinforcement Learning (Learning for Control)

Figure: (a) The controller aims to improve transient performances after disturbances while guaranteeing stabilization to the desired steady-state output $\bar{\bm{y}}$. (b) Two examples of physical systems with output tracking tasks. Exp1: Inverted pendulum track a required inverted angle $\Bar{y}$. Exp2:  A vehicle platoon tracks the setpoint velocity. (c) We provide end-to-end stability and output tracking guarantees by enforcing stabilizing PI structure in the design of neural networks. The key structure is strictly monotone functions, which are parameterized by the gradient of strictly convex neural networks. (d) The transient performance is optimized by training the neural networks. 

Communication-embedded Controller for Centralized, Distributed, and Decentralized Learning in Multi-agent Systems

UCSD DERConnect testbed: IEEE 13-bus simulation with RTDS simulator. 

Stability-constrained voltage control under high/low voltage disturbance.


[1] Yujia Huang, Ivan Dario Jimenez Rodriguez, Huan Zhang, Yuanyuan Shi, and Yisong Yue, "FI-ODE: Certifiably Robust Forward Invariance in Neural ODEs", Arxiv preprints, 2024.

[2] Wenqi Cui, Yan Jiang, Baosen Zhang, and Yuanyuan Shi, "Structured Neural-PI Control for Networked Systems: Stability and Steady-State Optimality Guarantees", in Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023. 

[3] Jie Feng, Yuanyuan Shi, Guannan Qu, Steven Low, Anima Anandkumar, and Adam Wierman, “Stability Constrained Reinforcement Learning for Real-Time Voltage Control in Distribution Systems”, IEEE Transactions on Control of Network Systems, 2023.

 [4] Jie Feng, Wenqi Cui, Jorge Cortes, and Yuanyuan Shi, “Bridging Transient and Steady-State Performance in Voltage Control: A Reinforcement Learning Approach with Safe Gradient Flow”, IEEE Control Systems Letters, 2023.

[5] Sahin Lale, Yuanyuan Shi, Guannan Qu, Kamyar Azizzadenesheli, Adam Wierman, and Anima Anandkumar, "KCRL: Krasovskii-Constrained Reinforcement Learning with Guaranteed Stability in Nonlinear Dynamical Systems", IEEE Conference on Decision and Control (CDC), 2023.

[6] Yuanyuan Shi, Guannan Qu, Steven Low, Anima Anandkumar, and Adam Wierman, “Stability Constrained Reinforcement Learning for Real-Time Voltage Control", in American Control Conference (ACC), 2022.


NSF EPCN 2200692

Early-Career Faculty Acceleration Award