Graph Attention Network-Driven Hierarchical Learning for Anti-Jamming UAV Communications

Jamming attacks pose a significant threat to the security of air-ground communications, where the challenge becomes more severe when involving multiple unmanned aerial vehicles (UAVs) incurring complex interference. To address this issue, this paper proposes a graph attention-based reinforcement learning strategy for anti-jamming UAV communications. Specifically, we consider the multi-UAV transmission and deployment in the presence of jamming attacks. Then, we formulate a zero-sum game with the legitimate side and adversary to maximize and minimize the overall transmission rate, respectively. Given the complicated structure of the game, we decompose it into two layers, tackled in a hierarchical learning framework. Particularly, the inner layer addresses the legitimate beamforming, for which we establish the graph attention network (GAT) to track the complicated interference and jamming relationship based on the graph representation of the UAV network. The outer layer address the legitimate UAV deployment and adversarial jamming policy, which is reinterpreted in a multi-agent deep reinforcement learning framework to obtain the strategies of both sides. The inner GAT is then nested within the outer multi-agent learning framework in a hierarchical manner to approximate the equilibrium of the original game model. Simulation results demonstrate the convergence and the performance superiority of the proposed learning scheme in terms of anti-jamming transmission rate. Also, the results exhibit significant generalization capability to cover different network configurations and parameters with reliable communication performance.