<p>3D Gaussian Splatting technique has achieved remarkable results in scene rendering. However, it still struggles to represent dynamic scenes with complexity, a high degree of freedom, and sparse viewpoints. To overcome these challenges, we introduce SpatioGS, an innovative method for real-time dynamic scene rendering that employs spatiotemporal-aware density control in Gaussian splatting. Our SpatioGS introduces an adaptive density control mechanism that dynamically adjusts the Gaussian distribution based on spatiotemporal features, enabling accurate capture of structural variations and fine texture details in complex motion regions. First, we propose a motion-aware keypoints mechanism, guided by spatiotemporal features, that isolates notable moving objects in the scene and promotes local density augmentation around them, thereby enhancing their structural representations. Secondly, we incorporate a pixel-wise error-guided pruning module to eliminate low-contribution Gaussians and noise, effectively minimizing computational redundancy and enhancing rendering efficiency. Thirdly, we propose a structural feature-anchor rigidity constraint that enhances local Gaussian consistency, thereby reducing motion artifacts in dynamic scenes. We assess the proposed SpatioGS using the HyperNeRF and Neu3D datasets and compare it with state-of-the-art methods. Experimental results demonstrate that our SpatioGS achieves 86 FPS at a resolution of <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(1352 \times 1014\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>1352</mn> <mo>×</mo> <mn>1014</mn> </mrow> </math></EquationSource> </InlineEquation> pixels and outperforms state-of-the-art methods in image quality. The source code is available at: <a href="https://github.com/caomw2/spatiogs">https://github.com/caomw2/spatiogs</a>.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SpatioGS: spatiotemporal-aware density control for dynamic scene rendering with Gaussian splatting

  • Mingwei Cao,
  • Zhihao Liu,
  • Ning Li,
  • Haifeng Zhao

摘要

3D Gaussian Splatting technique has achieved remarkable results in scene rendering. However, it still struggles to represent dynamic scenes with complexity, a high degree of freedom, and sparse viewpoints. To overcome these challenges, we introduce SpatioGS, an innovative method for real-time dynamic scene rendering that employs spatiotemporal-aware density control in Gaussian splatting. Our SpatioGS introduces an adaptive density control mechanism that dynamically adjusts the Gaussian distribution based on spatiotemporal features, enabling accurate capture of structural variations and fine texture details in complex motion regions. First, we propose a motion-aware keypoints mechanism, guided by spatiotemporal features, that isolates notable moving objects in the scene and promotes local density augmentation around them, thereby enhancing their structural representations. Secondly, we incorporate a pixel-wise error-guided pruning module to eliminate low-contribution Gaussians and noise, effectively minimizing computational redundancy and enhancing rendering efficiency. Thirdly, we propose a structural feature-anchor rigidity constraint that enhances local Gaussian consistency, thereby reducing motion artifacts in dynamic scenes. We assess the proposed SpatioGS using the HyperNeRF and Neu3D datasets and compare it with state-of-the-art methods. Experimental results demonstrate that our SpatioGS achieves 86 FPS at a resolution of \(1352 \times 1014\) 1352 × 1014 pixels and outperforms state-of-the-art methods in image quality. The source code is available at: https://github.com/caomw2/spatiogs.