Contingency-constrained economic dispatch with safe reinforcement learning

Overview

• Future power systems will rely heavily on microgrids with a high share of decentralized renewable energy sources and energy storage systems.
• The complexity and uncertainty in this context may make traditional power dispatch strategies impractical.
• Reinforcement learning (RL) controllers can address this challenge, but cannot themselves provide safety guarantees, preventing their deployment in practice.
• The researchers propose a formally validated RL controller for economic dispatch that addresses this limitation.

Plain English Explanation

The paper discusses how future power grids will increasingly rely on smaller, localized power systems called microgrids that use a lot of decentralized renewable energy sources like solar and wind, as well as energy storage. This creates a very complex and uncertain environment, making traditional ways of managing the power grid infeasible.

The researchers say reinforcement learning-based controllers could be a solution, as they can adapt to these dynamic conditions. However, these RL controllers can't guarantee safety, which prevents them from being used in real power systems.

To overcome this, the researchers developed an RL controller that has been formally verified to be safe. They added a special time-dependent constraint to the controller that accounts for the possibility of the microgrid becoming isolated (or "islanded") from the main grid. This constraint is calculated using a mathematical technique called set-based backwards reachability analysis. The RL agent's actions are then checked against this safety constraint, and any unsafe actions are adjusted to be safe before being implemented.

The researchers tested this approach on a residential microgrid case study using real-world data, demonstrating how it can provide safe and economical power dispatch in these complex future grid scenarios.

Technical Explanation

The researchers propose a formally validated reinforcement learning-based economic dispatch controller for microgrids with high penetration of decentralized renewable energy sources and energy storage.

They extend the conventional dispatch constraints with a time-dependent constraint that encodes the islanding contingency. This contingency constraint is computed using set-based backwards reachability analysis, which allows the controller to anticipate and avoid unsafe states.

The RL agent's actions are then verified through a safety layer that projects any unsafe actions into the safe action space. This leverages constrained zonotope set representations for computational efficiency.

The developed approach is demonstrated on a residential microgrid use case using real-world measurement data. The results show that the formally validated RL controller can provide safe and economically optimal power dispatch in these complex future grid scenarios.

Critical Analysis

The paper provides a promising solution to the challenge of enabling safe RL-based control in future power systems with high penetration of renewables and distributed energy resources. By incorporating formal verification techniques, the researchers are able to address the key limitation of standard RL controllers - the inability to provide safety guarantees.

However, the paper does not extensively discuss the potential computational complexity and scalability challenges of the proposed approach, especially as grid size and the number of distributed energy resources increase. There may also be practical implementation challenges in terms of obtaining accurate models and measurements required for the set-based reachability analysis.

Additionally, the paper focuses on economic dispatch optimization, but does not address other important aspects of microgrid control like voltage and frequency regulation, which would need to be considered for real-world deployment. Further research may be needed to extend this framework to handle a more comprehensive set of control objectives and constraints.

Overall, the work represents an important step towards enabling safe RL-based control in future power systems, but additional research and development will be required to fully realize this vision.

Conclusion

This paper presents a formally validated reinforcement learning-based controller for economic dispatch in microgrids with high penetration of decentralized renewable energy sources and energy storage. By incorporating a time-dependent constraint that encodes islanding contingencies, computed through set-based backwards reachability analysis, the approach is able to provide safety guarantees lacking in standard RL controllers.

The demonstrated residential microgrid case study shows the potential of this approach to enable safe and economically optimal power dispatch in complex future grid scenarios. While further research is needed to address scalability and broader control objectives, this work represents a significant advance towards the practical application of RL techniques in mission-critical power system applications.