Manipulating Constrained Soft Tissue while Avoiding Obstacles using Reinforcement Learning with Self-Attention

The complexity of dynamics raises challenges in deformable object manipulation, particularly the soft tissue shape control in in vivo environment. Previous studies assume the soft tissue is in an obstacle-free space, and the constraints are constant. To go one step further, a Soft Actor-Critic (SAC) algorithm is presented to manipulate the soft tissue under external disturbances while avoiding obstacles. K-Means is applied to reduce the dimension of the state space. Furthermore, self-attention is introduced to focus on the external forces and the constraints. Experiment results show that the agent can shape the soft tissue into the target deformation with the proposed approach under unknown external force, where the agent can automatically increase attention to the external force. The agent can find an obstacle-free path while manipulating the soft tissue. Compared with the SAC algorithm, the proposed approach with self-attention requires less training time, and the critic network is more stable.