Multi-agent deep reinforcement learning for two-level voltage/VAR control in active distribution networks considering component lifetime and heterogeneity

Converters of photovoltaic (PV) and battery energy storage system (BESS) can provide prompt and flexible reactive power support to voltage/var control (VVC) of distribution networks, but their lifetime can be significantly reduced due to additional reactive power output. This paper proposes a multi-objective two-level optimization algorithm to simultaneously improve voltage quality, enhances system economics, and extends component lifetime. Firstly, a multi-objective VVC optimization model is developed based on an integrated framework that incorporates detailed component lifetime models, explicit plant-internal structure model, and a lifetime-to-cost quantification method. Then, a two-level VVC algorithm is proposed. At the system level, multi-agent deep reinforcement learning (MADRL) coordinates distributed energy resources (DERs) to minimize voltage deviation and operating cost. Critically, at the plant level, a projected gradient descent (PGD) algorithm optimally reallocates power among heterogeneous internal units based on their distinct aging parameters, minimizing aggregate lifetime loss while respecting the system-level setpoints. At last, simulation results on a modified 141-bus distribution system with DERs verify that the proposed approach can effectively reduce voltage deviation, prolong the lifetime of converters and storage batteries, and minimize the overall operating costs, with sensitivity analysis yielding a Pareto front for operational trade-offs.