NIPM-sWMF: Toward Efficient FPGA Design for High-Definition Large-Disparity Stereo Matching

Large disparity stereo matching is critical to the application of a stereo vision system especially for outdoor scenes. Nevertheless, how to efficiently design high accuracy large-disparity stereo matching on a field-programmable gate array (FPGA) is still a grand challenge. The computational complexity of previously proposed stereo matching is inevitably proportional to disparity range; hence their hardware designs become very inefficient when the disparity range is large. Motivated by the original PatchMatch and weighted median filtering (WMF) algorithms, this paper proposes a non-iterative PatchMatch and separable WMF (NIPM-sWMF) algorithm to significantly reduce the computational complexity of stereo matching and make it independent of disparity range. Moreover, we also propose a fully pipelined architecture design on FPGA that employs several hardware techniques to efficiently implement the proposed NIPM-sWMF. The disparity quality of the proposed NIPM-sWMF algorithm is evaluated on both KITTI2015 and Middlebury V3 stereo data sets, and the proposed architecture design is implemented and synthesized on Xilinx FPGA. Evaluation results demonstrate that the proposed NIPM-sWMF design on FPGA reaches the real-time performance of 1920 × 1080@60 Hz at the disparity range of 128, and can achieve almost the same disparity estimation accuracy, 4.5× processing throughput, while reducing the hardware cost of LUT, Register, DSP, and BRAM by 40%, 47%, 100%, and 68%, respectively, compared with the reference stereo matching design. Therefore, the proposed NIPM-sWMF design is an efficient way to address the challenge of large-disparity stereo matching.