Evolutionary Game Based Demand Response Bidding Strategy for End-Users Using Q-Learning and Compound Differential Evolution

Load aggregators (LAs) play a key role in fully tapping the demand response (DR) resources of small and medium-sized end-users to enable a more flexible power grid. In the ancillary service market, the LA can provide DR to the system by aggregating the resources of its users. In response to the issued DR program, end-users offer to provide DR resources. To help optimize the user bidding strategy, an evolutionary game model is presented here in view of the bounded rationality of bidders. A combined Q-learning and compound differential evolution (CDE) algorithm is proposed to deal with the problems of incomplete information and uncertainties in the opponents' decision-making, and prevent the evolutionary stable strategy (ESS) from falling into a local optimum. Moreover, a cloud-computing-based framework is designed and agent servers are introduced to protect data privacy. Numerical results show that by adopting the proposed algorithm, the user's bidding price keeps slightly lower than the opponents' price which guarantees its revenue remains on a high level. This indicates that the proposed algorithm has good adaptability for addressing incomplete information and uncertainties in opponents' decision-making.