Adversarial learning for unguided single depth map completion of indoor scenes
摘要
Single depth map completion in the absence of any guidance from color images is a challenging, ill-posed problem in computer vision. Most of the conventional depth map completion approaches rely on information extracted from the corresponding color image and require heavy computations and optimization-based postprocessing functions, which cannot yield results in real time. Successful application of generative adversarial networks has led to significant progress in several computer vision problems including, color image inpainting. However, contrasting local and non-local features of depth maps compared to color images prevents the direct application of deep learning models designed for color image inpainting to depth map completion. Motivated by these challenges, in this work we propose to use deep adversarial learning to derive plausible estimates of missing depth information in a single degraded observation without any guidance from the corresponding RGB frame and any postprocessing. Different types of depth map degradations, such as simulated random and textual missing pixels as well as contiguous large holes found in Kinect depth maps, are effectively handled to reconstruct clean depth maps. An ablation study is also performed to investigate the contribution of our adversarial network architecture towards the recovery of missing scene depth information. We carry out an illustrative experimental analysis on the NYU-Depth V2 dataset and perform zero-shot generalization on the Middlebury and Matterport3D datasets, comparing our proposed method with several state-of-the-art algorithms. The experimental results demonstrate robustness and efficacy of the proposed approach.