ASPN: Adaptive Spatial Propagation Network for Depth Completion

Acquiring accurate pixel-wise scene depth maps is crucial to various visual applications such as autonomous driving and augmented reality. Most recent works are based on spatial propagation network (SPN). They generate initial dense depth maps and fixed affinity matrices for refining the former. However, the refinement process is rigid since the affinity matrices are fixed. For this purpose, we propose an adaptive spatial propagation network (ASPN) consisting of a state-aware (SA) module and a feature state-aware (FSA) module. SA can dynamically update the affinity matrices according to current depth completion states using a gated recurrent unit (GRU). SA improves the refinement flexibility and avoids the transmission of incorrect information in iterations. We further extend SA to feature levels as FSA, which expands the propagation receptive field and enhances the feature representation. Experiments on KITTI Depth Completion (DC) dataset and NYU Depth V2 dataset prove our ASPN balances volume and performance well. Specifically, ASPN requires only one-sixth of iterations and less than half the model parameters of our baseline method (NLSPN) to match or even exceed its performance.