Research on Complex Robot Manipulation Tasks Based on Hindsight Trust Region Policy Optimization

Deep reinforcement learning (DRL) algorithms have make remarkable progress in robot manipulation task in recent years. However, the success of completing the task relies heavily on the special design of reward function which requires engineering experience or domain-specific knowledge. To avoid complex reward shaping and make robot learning more general, it's of great essential to study the sparse-reward environments. In this paper, we present two types of challenging goal-conditioned sparse-reward tasks with 7-DoF robot arm, one is a target reaching task with obstacles, and the other is the dynamic object task where the target object moves at a certain speed. Based on the Hindsight Trust Region Policy Optimization (HTRPO) algorithm proposed by our research group, we studied the control performance on the two types of tasks with continuous high-dimensional state space. The results show that HTRPO can achieve more stable strategic performance, higher success rate and sample efficiency compared with its baseline algorithm TRPO and HPG. However, there still remains challenges in solving the tasks with high moving speed.