Multi-level object detection by multi-sensor perception of traffic scenes

Object detection is a hot research topic in the communities of computer vision and intelligent transportation system. In this paper, we propose a novel framework for multi-level object detection by mutli-sensor perception from road scenes. Firstly, two types of 2D object detection methods are proposed based on multi-level feature processing. Among them, an improved 2D object detection method is designed based on CenterNet, which introduces a centripetal offset module and a deformable convolution module to improve the corner matching accuracy and solve the problem of missed detection. Moreover, an improved 2D object detection method based on RetinaNet is designed by the optimization of the sub-network of ResNet. The coordinate attention mechanism and bidirectional feature fusion mechanism are incorporated. Finally, we propose a 3D object detection method based on an improved PointNet network. The point cloud data are projected into a frustum according to the 2D detection result, then the frustum is segmented according to the linearly increased steps. The experimental results on the KITTI dataset and TSD-max dataset well demonstrate the effectiveness of the proposed framework.