Deep Reinforcement Learning Based Preventive Maintenance for Wind Turbines

Wind turbines have been the core devices in the low carbon society, where the maintenance cost occupies the critical part of the life cycle cost. A preventive maintenance (PM) problem is formulated to reduce the maintenance cost caused by random failures of WTs, while guaranteeing the reliability of WTs with multiple components, e.g., blades, generators, main bearing, and gearboxes. A linear state transition function is introduced to capture the relation between the PM actions, degradation, and equivalent age of each component. The Weibull distributions and reliability functions are adopted to quantify the reliability level of each component. The state transition and reliability level functions are embedded as the environment function of a deep reinforcement learning (DRL) problem, where the maintenance costs caused by PM actions are treated as the reward function. The DRL problem is further formulated as a Q-network (DQN) with two hidden layers. A practical example is used to verify the effectiveness of the proposed strategy. The results indicate that the learned policy can take advantage of an opportunistic window to maintenance components whose condition is closed to be maintained, reducing the downtime duration and the maintenance costs.