DNOperNet: A Generic Network Architecture for Scalable and Safe Deep Reinforcement Learning in Distribution Network Operation

With the growing penetration of distributed energy resources (DERs) in distribution networks (DNs), the volatility and uncertainty make it increasingly challenging for DNs to operate. In addition, the safety constraints of these increasing number of controllable devices cannot be ignored, intensifying the challenge of coordinating them. To address these issues, this paper proposes a novel and generic network architecture for scalable and safe deep reinforcement learning (DRL) in DN operation, named DNOperNet. First, after formulating the DN operation problem into a Markov decision process (MDP), the impact of action dimensionality and safety concerns on the effectiveness of DN operation strategies is carefully analyzed. The proposed DNOperNet, featuring a branching architecture with safety masking, is then pertinently designed to effectively handle high-dimensional action spaces and safety constraints. Following this, a DNOperNet-based soft RL algorithm is specially developed. Case studies on a modified IEEE 33-bus network with many controllable devices prove the effectiveness the proposed method.