Graph Neural Network-Enhanced Multi-Agent Reinforcement Learning for Intelligent UAV Confrontation

Unmanned aerial vehicles (UAVs) are widely used in surveillance and combat for their efficiency and autonomy, whilst complex, dynamic environments challenge the modeling of inter-agent relations and information transmission. This research proposes a novel UAV tactical choice-making algorithm utilizing graph neural networks to tackle these challenges. The proposed algorithm employs a graph neural network to process the observed state information, the convolved output of which is then fed into a reconstructed critic network incorporating a Laplacian convolution kernel. This research first enhances the accuracy of obtaining unstable state information in hostile environments. The proposed algorithm uses this information to train a more precise critic network. In turn, this improved critic network guides the actor network to make decisions that better meet the needs of the battlefield. Coupled with a policy transfer mechanism, this architecture significantly enhances the decision-making efficiency and environmental adaptability within the multi-agent system. Results from the experiments show that the average effectiveness of the proposed algorithm across the six planned scenarios is 97.4%, surpassing the baseline by 23.4%. In addition, the integration of transfer learning makes the network convergence speed three times faster than that of the baseline algorithm. This algorithm effectively improves the information transmission efficiency between the environment and the UAV and provides strong support for UAV formation combat.