Learning-based Optimal Formation Scheme for Disturbed Multi-agent Systems via Generalized Policy Iteration

In order to solve the formation problem of nonlinear continuous-time multi-agent system under disturbance injection and input saturation, a robust formation scheme is designed based on generalized policy iteration and adaptive critic learning. First, the robust formation problem is depicted as a mixed game. In this case, a cooperative game is established among the agents, and the local neighborhood formation error stabilization problem of each agent under disturbance is described as zero-sum differential game problem, the key to which is to solve the Hamilton-Jacobi-Isaacs equation. Then, to obtain the approximate solution of Hamilton-Jacobi-Isaacs equation, a continuous-time generalized policy iteration algorithm is designed. In order to implement the iterative algorithm, the single-critic neural network structure is employed to obtain the optimal formation policy under the worst-case disturbance. It is proved that the local neighborhood formation error and the weight estimation error of the critic network are uniformly ultimately bounded. Finally, the proposed learning-based robust formation scheme is applied to a nonlinear multi-agent system and a spacecraft formation system. Simulation results demonstrate that the proposed robust formation scheme has good cognition and suppression ability to disturbance injection, so as to complete the formation task.