Implicit neural rendering excels in modeling dynamic scenes for novel view synthesis (NVS) but struggles with real-time rendering and storage efficiency. While 3D Gaussian Splatting (3D-GS) addresses these limitations through explicit representation, the implementation of this method in dynamic environments continues to present difficulties, primarily stemming from the requirement for high-fidelity Structure-from-Motion (SFM) initialization and difficulties in handling occlusions and view-dependent inconsistencies. To address the challenges, we propose a novel method for dynamic scene reconstruction called Dual-Gradient Dynamic Splatting, a novel method that learns dynamic Gaussians within a deformable field and density gradients emerge from two photometric constraints: (a) view-dependent projection extents of Gaussian splats, and (b) proximity metrics relative to the geometric center of the reconstructed volume. By incorporating pixel and distance information as dual-gradient into density control, our method is able to overcome the problem of high dependence on SFM. Additionally, we leverage pre-trained depth estimation models to provide depth cues, further guiding the training of the Gaussians to better fit the geometric structure of the moving object. Experiments demonstrate that our approach achieves real-time, high-fidelity dynamic rendering in monocular scenes, even with degraded initial point clouds. It outperforms existing methods in reconstructing intricate details and dynamic objects, offering a robust solution for dynamic scene reconstruction.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dual-Gradient Dynamic Splatting: Density Control with Dual-Gradient for 3D Gaussian Splatting in Monocular Dynamic Scene

  • Wen Shen,
  • Hao Tian,
  • Jiacen Liu,
  • Xiaolin Qin

摘要

Implicit neural rendering excels in modeling dynamic scenes for novel view synthesis (NVS) but struggles with real-time rendering and storage efficiency. While 3D Gaussian Splatting (3D-GS) addresses these limitations through explicit representation, the implementation of this method in dynamic environments continues to present difficulties, primarily stemming from the requirement for high-fidelity Structure-from-Motion (SFM) initialization and difficulties in handling occlusions and view-dependent inconsistencies. To address the challenges, we propose a novel method for dynamic scene reconstruction called Dual-Gradient Dynamic Splatting, a novel method that learns dynamic Gaussians within a deformable field and density gradients emerge from two photometric constraints: (a) view-dependent projection extents of Gaussian splats, and (b) proximity metrics relative to the geometric center of the reconstructed volume. By incorporating pixel and distance information as dual-gradient into density control, our method is able to overcome the problem of high dependence on SFM. Additionally, we leverage pre-trained depth estimation models to provide depth cues, further guiding the training of the Gaussians to better fit the geometric structure of the moving object. Experiments demonstrate that our approach achieves real-time, high-fidelity dynamic rendering in monocular scenes, even with degraded initial point clouds. It outperforms existing methods in reconstructing intricate details and dynamic objects, offering a robust solution for dynamic scene reconstruction.