Learning general multi-agent decision model through multi-task pre-training

Multi-agent reinforcement learning (MARL), involving collaboration and competition among multiple agents, demonstrates significant potential in addressing complex tasks and facilitating the cooperative nature of artificial intelligence systems. However, training MARL networks is a challenging task, demanding extensive computational resources to interact with complex environments, extract state representations, and acquire decision knowledge. Recent breakthroughs in large-scale pre-trained models have sparked our interest. We aim to discover shared knowledge in MARL through model pre-training. In multi-agent systems, the absence of shared state representations often arises due to differences in environments across tasks. This complicates the pre-training of common perception models. Nevertheless, there typically exists common decision logic among different multi-agent tasks, such as cooperation. Pre-training a general decision model for multi-agents proves to be an efficient solution. However, the coupling of perception and decision-making often impedes the individual pre-training of decision models. In this paper, we propose a novel approach based on multi-task pre-training to learn a general multi-agent decision model (GMADM). This approach decouples perception from decision-making, extracting shared decision logic across different tasks, enabling rapid learning of excellent policies on new tasks. Our approach demonstrates outstanding performance in the StarCraft Multi-Agent Challenge (SMAC) and Google Research Football (GRF) environments. Experimental results indicate that pre-trained general decision models can smoothly transfer to new tasks, significantly reducing training costs and improving performance. Additionally, visualization confirms the presence of common multi-agent decision knowledge in shared network layers, further validating the effectiveness of our approach. Code is available at https://github.com/wangjw55/GMADM.