An Adaptive Interpretable Safe-RL Approach for Addressing Smart Grid Supply-Side Uncertainties

Integration of large-scale renewable energy sources is the future of smart grids. However, weather conditions significantly affect the generation of renewable energy sources, which causes supply-side uncertainties. Even with the current state-of-the-art technologies, weather conditions, solar radiance, etc., can be predicted nearly accurately, only a few minutes early. Existing intelligent grid optimization methods used to solve Optimal Power Flow (OPF) consider day-ahead planning for congestion management and generator rescheduling. Current OPF-based methods suffer many drawbacks for the large grid, like longer computation time and local convergence issues, and are computationally expensive. This drawback makes existing OPF methods unsuitable for the congestion management problem when considering supply-side uncertainties, mainly due to the large-scale integration of renewable energy sources. Over the years, Reinforcement Learning (RL) methods have gained popularity for solving constrained optimization tasks at scale. However, even if the existing Deep-RL (DRL) based policies perform best for large and complex tasks, the lack of interpretability due to using deep neural networks at their core makes them not a trustworthy solution when safety constraints like congestion-free operation are considered. Therefore, to ensure that DRL policies are used safely in critical infrastructures like smart grids, this paper proposes a centralized barrier-penalty-based DRL method that utilizes the Adaptive Safety Shield framework as an extra layer of safety. The DRL-based policy is trained to find the economically optimal grid state while considering the barrier penalty value to reduce branch loads. Besides, the Adaptive Safety Shield is trained to learn the congestion criteria and guide the RL in avoiding such scenarios during exploration. We have also empirically shown that the proposed method has demonstrated similar or better performance than the current OPF-based methods.