An FPGA-based Real-Time Optical Flow Accelerator for Recurrent All-Pairs Field Transforms

Optical flow can capture the positional changes of pixels between two consecutive frames, thereby enabling the extraction of motion information for objects. Real-time optical flow estimation is widely applied in tasks such as motion estimation, object detection, and tracking. Deep neural network-based optical flow algorithms have made a significant breakthrough in accuracy compared to traditional methods; however, their dense computational requirements hinder real-time deployment on resource-constrained embedded platforms. In this paper, we present ERAFT: a novel lightweight deep neural network architecture based on the Recurrent All-Pairs Field Transforms (RAFT) algorithm, which is more suitable for hardware deployment. Furthermore, we propose a specialized optical flow accelerator based on a prediction mechanism, enabling real-time and efficient optical flow estimation computations. The hardware accelerator was evaluated on the Xilinx VCK190 evaluation board. The results indicate that this accelerator achieves high accuracy on the Middlebury dataset, with speeds of up to 86 frames/s for 640 × 480 pixel images.