Data-driven Power Systems Control with Stability Guarantee

Power systems are experiencing a period of rapid changes due to the proliferation of renewable generation and distributed energy resources including rooftop solar, electric vehicles, and batteries. Many of these new technologies are interfaced with the grid through power electronic interfaces (i.e., inverters) that can be controlled at a much faster timescale compared to conventional machines. However, how to leverage such flexibility is nontrivial due to the nonlinearity, complexity, and uncertainty in the grids. 

To overcome these challenges, our research develops a new data-driven control framework for power system voltage and frequency control with formal stability guarantees. The main tool we use is the Lyapunov stability theory, which is a rich topic studied in the control community and a wealth of results on Lyapunov and energy functions for power system control. However, these techniques were not studied with reinforcement learning (RL) and neural networks in mind. We propose a stability-constrained RL framework that bridges the flexibility of neural networks, the ease of optimization provided by RL, and the stability guarantees certified by Lyapunov functions. 

See below for some recent publications related to this project.

Stability Constrained Reinforcement Learning for Real-Time Voltage Control

Deep reinforcement learning (RL) has been recognized as a promising tool to address the challenges in real-time control of power systems. However, its deployment in real-world power systems has been hindered by a lack of formal stability and safety guarantees. In this paper, we propose a stability-constrained RL method for real-time voltage control in distribution grids and we prove that the proposed approach provides a formal voltage stability guarantee. The key idea underlying our approach is an explicitly constructed Lyapunov function that certifies stability. The proposed method can reduce the transient control cost by more than 30% and shorten the response time by a third compared to a widely used linear policy, while always achieving voltage stability. In contrast, standard RL methods often fail to achieve voltage stability.

Sponsers

NSF EPCN 2200692