Depth Map Super-Resolution via Asymmetrically Guided Feature Selection and Spatial Affine Transformation

Guided depth image super-resolution (GDSR) is a fundamental task that involves reconstructing a low-resolution (LR) depth image to a high-resolution (HR) version using an aligned HR color image as guidance. The effective utilization of the HR color image to guide the depth image super-resolution process is a critical consideration in algorithm design. In this paper, we propose a novel asymmetric channel-spatial fusion (ACSF) module to address this concern. Specifically, the depth feature is iteratively refined by corresponding color feature in both channel and space dimensions with ACSF during feature extraction. For channel dimension, we utilize asymmetric channel attention to achieve the goal of feature selection, and affine transformation is applied in the space dimension to modulate the depth feature with the guidance color feature. The resulting deep depth map features, which comprehensively integrate the information from RGB modal, are then employed to reconstruct high-resolution depth maps. In other words, The asymmetry is reflected in the fact that we only explicitly update the depth features. The effectiveness of our approach is demonstrated through quantitative and qualitative experiments. The visualization results indicate that the ACSF module enables the network to focus more on areas with larger errors in the super-resolution results, typically corresponding to object edges with sharp changes in the depth map. In terms of quantitative evaluation, our proposed method achieves new state-of-the-art (SOTA) on three widely used benchmarks under different scales.