Dual Branch Image-Guided Network with Multi-stage Iterative Refinement for Depth Completion

Image-guided depth completion focuses on predicting dense depth information using its sparse measurement and the corresponding aligned RGB image. Existing deep learning-based networks do not fully leverage the advantages of convolution and Transformer for feature processing, thereby limiting the estimation performance of the network. This paper proposes an image-guided network with multi-stage recurrent refinement for depth completion. The network consists of a coarse depth estimation branch and a depth refinement branch. The coarse depth estimation branch incorporates a hybrid Transformer structure that combines the benefits of convolution and Transformer in a complementary manner to estimate a coarse depth. The depth refinement branch efficiently fused the context features extracted by a convolutional backbone with the hybrid Transformer features using a cross self-attention mechanism. The proposed multi-stage iterative refinement module progressively refines the features to obtain the target depth map. Effectiveness of our method is demonstrated by comparing with several state-of-the-art networks on KITTI benchmark dataset.