Multiscale Adaptation Fusion Networks for Depth Completion

Depth completion is becoming a particularly important yet challenging problem with the growingly rapid progress of depth sensing technologies. Depth completion aims to complete sparse and noisy depth images to generate dense depth images. In this paper, we propose a multiscale adaptation fusion network (MAFN) for depth completion. The depth features are fused with RGB features at multiple scales with adaptation modules, where a neighbour attention mechanism is designed to adapt the local structures of the RGB image and the depth image. The fusion and completion process are unified under the encoder-decoder framework which is learned in an end-to-end way. By exploiting the detailed structural relationships of RGB images and depth images, our MAFN model can accurately complete and restore the invalid depth values on the sparse depth images. We test the proposed method on the challenging KITTI depth completion benchmark. The experimental results prove the effectiveness and strength of the proposed method.