Vision-based Robotic Grasp Success Determination with Convolutional Neural Network

We present two Convolutional Neural Network (CNN) models to determine whether a robot successfully grasps the target object using RGB image as input. As we know, this is the first time that vision is applied to help robot determine if something is grasped by itself. Our network performs end-to-end classification on our own dataset collected using Baxter Robot. This requires the network to acquire the refined spatial relationship between gripper and target in the image. Compared with traditional ways of using only force and distance sensors, which are restricted in terms of object size, weight and grasp postures, vision-based CNN models have potential ability to be applied in wider range and are more similar to human ways in aspect of perceiving environment. The experimental result shows that it is feasible to use images to help robot determine whether the target is grasped to overcome the drawbacks of only using force or distance sensors. In addition, we also test our networks on Baxter Robot, the results show that our models have good generalization ability and can fit the situation of real world well.