Deep Reinforcement Learning-Based Near-Field Channel Estimation for Extremely Large-Scale MIMO Systems

The extremely large-scale array is considered to be one of the key technologies for 6G, which can significantly improve spectral efficiency. However, the extremely-large number of antennas results in a larger range of the near field (e.g., hundreds of meters), leading to the electromagnetic wave propagation modeling changing from plane wave to spherical wave. This makes conventional channel estimation methods suffer from inevitable performance degradation due to the additional distance information in the spherical wavefront. To address this problem, this paper proposes a deep reinforcement learning based near-field channel estimation, in which the multi-agent deep deterministic policy gradient (MADDPG) is employed. Specifically, the near-field channel estimation task is first formulated as a compressed sensing problem by using a sparse spatial grid-based dictionary. Then, the Actor Critic (AC) network based MADDPG algorithm is employed to jointly optimize the angular and distance information by an approximate global search. In addition, an advantage Actor Critic network-based channel estimation algorithm is proposed, which improves both the stability and efficiency of the AC-based algorithm. Finally, the numerical results show that the proposed algorithms outperform the benchmarks in terms of normalized mean square error.