Visual localization of robotic end effector via fusion of 3D Gaussian Splatting and heuristic optimization algorithm

The visual localization of the robotic end effector holds significant importance and value in intelligent manufacturing. Regarding this issue, a pose estimation framework integrating the radiance field rendering and an improved differential evolution algorithm is proposed in this paper. Firstly, multi-view images of the object are collected by an industrial camera carried on the robotic arm. Then, a 3D Gaussian Splatting model is trained, and the observation trajectory of the virtual camera is designed based on the query image in Nerfstudio for visualization. The key frames and the corresponding pose matrices of the rendered videos along the trajectory are extracted. Subsequently, the improved differential evolution algorithm through double-layer optimization proposed in this paper is employed to conduct similarity matching between the query image and the extracted rendered images. The pose matrix corresponding to the most similar rendered image is the estimated pose of the query image. Through experiments, this paper explores the influence of the frame sampling rate on the pose estimation accuracy, and conducts comparative and ablation experiments with the commonly used methods at present. The experimental results demonstrate that the proposed method possesses good feasibility and outstanding performance.