D4Recon: Dual-Stage Deformation and Dual-Scale Depth Guidance for Endoscopic Reconstruction
摘要
Deformable tissue reconstruction in endoscopy is vital for surgery, yet current methods struggle with high-fidelity reconstruction of irreversible tissue deformations. To this end, we present D4Recon, a novel framework for real-time and high-fidelity endoscopic reconstruction, addressing crucial challenges in surgical applications. A ual-stage eformation modeling and a ual-scale epth guidance ( ) are proposed in a dynamic 3D Gaussian Splatting paradigm along with lightweight multi-layer perception (MLP) to model dynamics in endoscopic scenes. In the dual-stage deformation modeling, we introduce a spatial deformation model to correct multiview inconsistencies, accompanied by a temporal deformation model that accurately represents tissue distortion and dynamic tissue interaction with surgical tools in the reference frames. In the dual-scale depth guidance, we propose to balance local error correction with absolute depth consistency, enabling precise depth refinement while preserving fine-grained color accuracy. D4Recon generates accurate 3D reconstructions with superior PSNR, SSIM, and LPIPS scores, outperforming existing methods in terms of geometric coherence and photorealism with real-time rendering speed, as demonstrated by extensive experiments on diverse endoscopic datasets. Reconstruction videos are in the supplementary file. Website .