Underwater Multi-agent Cooperative Formation Hunting Based on Deep Reinforcement Learning

In addressing the issue of formation hunting and trajectory planning for multi-autonomous underwater vehicles (AUVs) in complex underwater environments, traditional virtual structure algorithms, and leader-follower models exhibit shortcomings in environmental adaptability and vulnerability to single-point failures. To solve this problem, this article establishes a multi-agent reinforcement learning model with continuous state and action spaces, aiming to optimize the success rate and completion time of the formation hunting task. Furthermore, in establishing the simulation environment for underwater multi-AUVs, a reward function module for the formation hunting task is meticulously designed, taking into account various factors including navigation, formation, efficiency, boundary, and collision avoidance. The efficacy of the proposed methodology was substantiated through a comparative analysis involving the artificial potential field method and the proposed deep reinforcement learning algorithm within the simulation environment. Besides, the efficiency of task execution has improved by approximately 10%, with a success rate approaching 100%.