Real-Time Operation Management for Battery Swapping-Charging System via Multi-Agent Deep Reinforcement Learning

Battery swapping-charging systems (BSCSs) can provide better battery swapping services for electric vehicles (EVs) in large cities. In BSCSs, EV batteries can be centrally charged at battery charging stations (BCSs) and then dispatched via delivery trucks to battery swapping stations (BSSs) to support local EVs. This paper considers the real-time optimization scheduling problem in BSCS, including battery charging, swapping and truck routing. We model this real-time scheduling problem as a decentralized partially observable Markov decision process (Dec-POMDP) and solve it using multi-agent deep reinforcement learning (MADRL) algorithms. The joint scheduling process of trucks and BCSs has many dynamic hard constraints between them that cannot be solved using the existing MADRL algorithms. To this end, we combine MADRL with binary integer programming (BLP) and propose the Value Decomposition Network (VDN)-BLP algorithm to solve the problem with constraints. We also combine actor-critic architecture and local search with VDN-BLP to substantially improve computational efficiency with little performance loss. Simulation results based on historical battery swapping data in Sanya City verify the effectiveness of the proposed method.