Optimal Energy Scheduling for Microgrids with a Hybrid Battery-Hydrogen Storage System Based on Reinforcement Learning

This paper explores the integration of battery and hydrogen storage in a Microgrid (MG), combining the high-power capabilities of battery with the high-capacity characteristics of hydrogen storage to manage instantaneous load variations and balance peak-valley energy differences. The scheduling method employs the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, recognizing the complexity of the integrated energy system. 24-hour simulations are conducted for representative days across the four seasons - spring, summer, autumn, and winter. The TD3 algorithm minimizes fluctuations in power exchanges with the main grid by effectively synchronizing the operations of battery and Hydrogen Storage System (HSS), thus enhancing grid stability. It learns the patterns between load demands and renewable energy generation, favoring battery charging and hydrogen production during low demand periods and discharging or utilizing hydrogen for electricity during peak demand. This strategy promotes more effective utilization of renewable resources. Additionally, a comparison with the Deep Deterministic Policy Gradient (DDPG) algorithm shows that while the cost-related performance for both TD3 and DDPG is nearly identical, TD3 demonstrates significantly better stability, with standard deviations consistently lower than those of DDPG. In particular, under winter conditions, the standard deviation of TD3 is only 45.5% of that of DDPG.