Adaptive Link State Update Scheme for Large-Scale LEO Satellite Networks Based on Distributed Deep Reinforcement Learning

In the upcoming sixth generation (6G) era, dynamic routing relying on link state information update is crucial for global data service in large-scale low-earth orbit (LEO) satellite networks. However, the existing dynamic routing methods use a static link state update scheme where all satellites distribute their link state information with the same fixed period, while the link state of the satellites are different and vary dynamically. This makes it difficult to achieve the balance among various network performance metrics such as link state update accuracy, signaling overhead, network throughput, and energy efficiency. To solve this issue, we propose an adaptive link state update scheme for the LEO satellite network, where each satellite can dynamically adjust its own link state distribution interval according to the observation on the inter satellite links (ISLs). Based on the proposed scheme, we define the information deviation to characterize the accuracy of the link state update and derive the signaling overhead of link state distribution. To improve further the network performance, a multi-objective optimization problem (MOP) is formulated to minimize the information deviation and the signaling overhead simultaneously. By applying the weighted sum method, we convert the formulated MOP into a single-objective optimization problem (SOP). Then, we adopt the distributed reinforcement learning approach and develop the deep Q-network (DQN) algorithm for each satellite to learn its optimal link state distribution decision strategy based on local information. Simulation results demonstrate the superiority of the proposed scheme.