Multi-Agent Q-Value Mixing Network with Covariance Matrix Adaptation Strategy for the Voltage Regulation Problem

Control and optimization of power systems typically involves schemes that utilize optimal power flow techniques and comprehensive modeling of various electrical components. However, the extensive integration of renewable energy sources and distributed energy resources makes it difficult to obtain accurate models, making the traditional model-based approaches more challenging. In this paper, we propose a Covariance Matrix Adaptation Q-value network mixing method (CMAQMIX), which is a novel model-free multi-agent reinforcement learning method that combines the advantages of Covariance Matrix Adaptation Evolution Strategy and the Q-value network mixing method to solve the problem of continuous action space. We establish a new multi-agent voltage regulation environment based on CityLearn framework to test the CMAQMIX method. The results show that our proposed method outperforms the Independent Proximal Policy Optimization method and can give immediate action response without the complete domain knowledge of the power system.