Pre-Trained Actor-Guided Proximal Policy Optimization for Feasible and Safe AC Optimal Power Flow in Large-Scale Power Systems

Real-time AC Optimal Power Flow (AC-OPF) optimization poses a critical challenge in modern power systems, due to the computational complexity of traditional solvers and the infeasibility issues associated with the reinforcement learning (RL) methods. We propose Pre-trained Actor-Guided Proximal Policy Optimization (PAG-PPO). This framework initializes an RL agent with the actor neural network cloning from historical OPF solutions to enhance learning efficiency. This approach enables rapid convergence to physically feasible and numerically stable control policies, ensuring consistent power flow convergence while minimizing feasible action exploration in the training. Evaluated on the IEEE 300-bus system under diverse load conditions and line outages, PAG-PPO achieves power flow convergence in 500 episodes. Compared to the MATPOWER solver, PAG-PPO reduces inference time by 114×, with a 0.79% increase in generation cost and a 99.6% feasibility rate. Infeasible cases involve minor voltage violations, with no power flow divergence. These results demonstrate that PAG-PPO offers a practical, scalable, and near-optimal solution for real-time AC-OPF optimization, advancing RL-based decision-making for large-scale power systems.