<p>Point cloud data has become increasingly important in applications ranging from autonomous driving to virtual reality. However, raw point clouds acquired from challenging environments or complex scenes often suffer from sparsity. Densifying the sparse point clouds can boost downstream tasks, such as object recognition and 3D reconstruction. This paper proposes a novel point cloud upsampling method based on 3D Gaussian Splatting (3DGS), leveraging differentiable rendering to optimize newly interpolated points. Our approach creates a mesh from the raw point cloud and interpolates new points on the triangle surfaces. The position of each new point is parameterized by two triangle edge weights, which are optimized through training the 3DGS representations. The method combines the position-optimized new points with the raw input point cloud thereby achieving upsampling. We also investigated the impact of a depth regularization loss during 3DGS refinement. Experiments on the DTU dataset demonstrate the effectiveness of our mesh-based point cloud upsampling method with 3DGS optimization in terms of accuracy and completeness metrics. Compared with existing MLS-based and deep-learning methods, our approach reduces the average completeness error by 0.019 mm while maintaining geometric stability, offering a promising direction for addressing point cloud sparsity in 3D data processing and computer vision applications. Our approach is implemented and released at: <a href="https://github.com/zichen34/mesh-pu-w-3dgs">https://github.com/zichen34/mesh-pu-w-3dgs</a></p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Mesh-based point cloud upsampling with 3D Gaussian splatting

  • Zichen Wang,
  • Ning Zhang,
  • Xiaoyan Wang,
  • Q. M. Jonathan Wu

摘要

Point cloud data has become increasingly important in applications ranging from autonomous driving to virtual reality. However, raw point clouds acquired from challenging environments or complex scenes often suffer from sparsity. Densifying the sparse point clouds can boost downstream tasks, such as object recognition and 3D reconstruction. This paper proposes a novel point cloud upsampling method based on 3D Gaussian Splatting (3DGS), leveraging differentiable rendering to optimize newly interpolated points. Our approach creates a mesh from the raw point cloud and interpolates new points on the triangle surfaces. The position of each new point is parameterized by two triangle edge weights, which are optimized through training the 3DGS representations. The method combines the position-optimized new points with the raw input point cloud thereby achieving upsampling. We also investigated the impact of a depth regularization loss during 3DGS refinement. Experiments on the DTU dataset demonstrate the effectiveness of our mesh-based point cloud upsampling method with 3DGS optimization in terms of accuracy and completeness metrics. Compared with existing MLS-based and deep-learning methods, our approach reduces the average completeness error by 0.019 mm while maintaining geometric stability, offering a promising direction for addressing point cloud sparsity in 3D data processing and computer vision applications. Our approach is implemented and released at: https://github.com/zichen34/mesh-pu-w-3dgs