TD3-Based Model Predictive Control for Satellite Formation-Keeping

The escalating prevalence of formation flights in space missions has led researchers to intensify their focus on designing optimal control systems for satellite formation motion along reference orbits, with the aim of reducing tracking error and energy consumption. However, conventional controllers typically excel at optimizing only one of these objectives, and the manual parameter tuning of such controllers proves to be a challenging task. In this paper, we introduce a novel approach, the twin delayed deep deterministic policy gradient-based model predictive control (TD3-MPC) method. To tackle the multiobjective formation-keeping challenge, a linear model predictive controller based on the satellite's dynamics had been developed. Subsequently, a cost function is formulated to facilitate the optimization of multiple objectives, specifically tracking error and fuel consumption. In addressing the intricate issue of controller parameter tuning, we employ reinforcement learning and design a reward function reflective of the TD3 algorithm's controller performance. Simulation results underscore the superior performance of the proposed TD3-MPC algorithm compared to the linear model predictive controller, achieving a notable 27.83% reduction in tracking error and a substantial 48.30% decrease in fuel consumption under large error condition and 3.67% reduction in tracking error and a substantial 22.27% decrease in fuel consumption under small error condition. By effectively combining the strengths of reinforcement learning and model predictive control, TD3-MPC enhances the satellite's ability to adhere more precisely to its intended trajectory, thereby ensuring the stability and desired operational performance of the satellite formation.