Learning A Deep Second-Order Unfolding Model for Arbitrary-Scale Depth Map Super-Resolution

Depth map Super-Resolution (DSR) is a pivotal foundation for spatial computing in virtual-real fusion scenarios, with deep unfolding networks being a promising research approach. However, most existing deep unfolding methods rely on first-order optimization algorithms, suffering from slow convergence and sub-optimal learning efficiency. To tackle these issues, we propose a Second-order semi-smooth Newton Unfolding Network (SNU-Net) for arbitrary-scale DSR, which is formed by expanding DSR optimization model via the second-order semi-smooth Newton algorithm. SNU-Net adopts a two-stage progressive reconstruction framework, comprising two main parts: augmented Lagrange multiplier update network and Depth map reconstruction Network (DNet) which is build upon First-order reconstruction Network (FNet) and Second-order reconstruction Network (SNet). Here, the FNet handles initial feature mapping and shallow structure recovery, while the SNet conducts multi-scale feature transformation and fusion to progressively refine depth features. To achieve arbitrary-scale DSR, we introduce a decoupled semi-scale sampling mechanism to resolve structural information loss in previous unfolding networks caused by the mapping relationships at different resolutions merely involving a single-stage up and down sampling process. This mechanism divides the up-sampling process into two consecutive sub-stages: semi-scale sampling and fine sampling. Comprehensive experiments have demonstrated that the proposed SNU-Net outperforms several methods, achieving superior reconstruction fidelity and structural consistency.