Double Deep Q-learning Based on Personalized Thermal Comfort Model for HVAC Optimization

The operation of Heating, Ventilation and AirConditioning (HVAC) systems in buildings has huge energy saving potential and therefore HVAC optimization can greatly reduce carbon emission. How to balance energy cost and thermal comfort of occupants still needs to be researched. The most common approach is to use a static range of air temperature or PMV(Predicted Mean Vote) to describe thermal sensation and regard it as constraints for energy optimization problem. However, further research illustrates that people may perceive differently in the same environment, and its effect on HVAC control has not been analysed. To address this problem, the personalized thermal comfort is considered to further improve energy efficiency and satisfaction of occupants. Specifically, Double Deep Q-learning based on Personalized Thermal Comfort model for HVAC optimization(called PTCDDQ framework) is proposed in this work. First, metabolic rate is used to describe thermal difference, and it is estimated by genetic algorithm using actual votes of occupants. Second, PMV thermal models with different metabolic rates are combined with HVAC models simulated by Energyplus to formulate the optimization problem. Then Double Deep Q-learning algorithm is applied to solve the problem. Third, three kinds of people, coldintolerant, neutral and hot-intolerant are defined to compare the performance of PTCDDQ and traditional control methods. Case study results show that PTCDDQ framework can enhance energy efficiency and thermal satisfaction at the same time.