Digital twin-empowered robotic arm manipulation with reinforcement learning: A comprehensive survey

Recent decades have witnessed rapid development and increasing widespread applications of robotics across various industries. On one hand, the robotic arm, being the key component of robotics, has attracted the attention of scholars and experts with its application in quite a number of smart factory tasks. On the other hand, Digital Twin (DT), as an emerging virtual-physical bridging technique, offers significant advantages over testing robotic arm manipulation algorithms only within simulation environments. By facilitating the accurate validation of algorithms in real environments, DT provides a realistic basis for testing and optimizing their feasibility. This paper discusses the state-of-the-art of robotic arm intelligent manipulation related techniques empowered by DT and illustrates the picture for its future development. More specifically, it provides a novel perspective to analyze the entire workflow of DT-empowered robotic arm intelligent manipulation techniques, from task definition to path planning, simulation environment, and virtual-real communications, respectively. First, diverse robotic arm manipulation tasks, such as catching, picking & placing, and assembling are reviewed along with the methods of path planning and collision avoidance. Second, this paper discusses the evolution of various path planning algorithms for robotic arm manipulation, highlighting reinforcement learning methods such as Deep Q-learning and Proximal Policy Optimization approaches. Third, this paper reviews on the simulation environments containing Unity, MuJoCo, ROS, PyBullet and so on, in which different deep learning methods are implemented. Finally, recent developed robotic arm DT systems including some new Augmented Reality and Virtual Reality aided applications are analyzed. It is hoped that this study will provide valuable insights for DT-empowered robotic arm techniques and pave the way for further development of more advanced researches.