Memory-Augmented Reinforcement Learning for Secondary Control of Microgrids in Cyber Attacks and Nonideal Operating Conditions

With large amounts of renewable energy, typical cyber attacks and nonideal operating conditions coexisting, the adaptive ability of the microgrid control based on physical models is insufficient due to the heavy reliance on specific optimization models, which need to be smart in different situations. This article proposes a memory-augmented reinforcement learning framework, in which the state-adaptive input weighting mechanism is innovatively developed to dynamically determine the significance of different state components, and then the recurrent neural network with long sequence processing capability is embedded to capture the long-term state dependencies, thereby augmenting memory and situation awareness capability. The secondary control problem is modeled as a Markov decision process, in which the knowledge-assisted reward design, integrated with staged training and adjusted exploration, jointly enhances the robustness and adaptability of microgrid secondary control. Simulation results of four cases show that the proposed control strategy significantly reduces the system frequency and voltage fluctuations. Compared with other strategies, it performs the best in terms of standard deviation, average deviation and excellent rate indicators.