Deep Arbitrary-Scale Unfolding Network for Color-Guided Depth Map Super-Resolution

Although color-guided Depth map Super-Resolution (DSR) task has made great progress with the help of deep learning, this task still suffers from some issues: 1) many DSR networks are short of good interpretability; 2) most of the popular DSR methods cannot achieve arbitrary-scale up-sampling for practical applications; 3) dual-modality gaps between color image and depth map may give rise to texture-copying problem. As for these problems, we build a new joint optimization model for two tasks of high-low frequency decomposition and arbitrary-scale DSR. According to alternatively-iterative update formulas of the solution for these two tasks, the proposed model is unfolded as Deep Arbitrary-Scale Unfolding Network (DASU-Net). In the DASU-Net, we propose a Continuous Up-Sampling Fusion (CUSF) module to address two problems of arbitrary-scale feature up-sampling and dual-modality inconsistency during color-depth feature fusion. A large number of experiments have demonstrated that the proposed DASU-Net achieves more significant reconstruction results as compared with several state-of-the-art methods.