Scanning objects with modern registration devices typically gives incomplete point clouds, primarily due to the limitations of partial scanning, 3D occlusions, and dynamic light conditions. Recent works on processing incomplete point clouds have always focused on point cloud completion. However, these approaches do not ensure consistency between the completed point cloud and the captured images regarding color and geometry. We propose using Generative Point-based NeRF (GPN) to reconstruct and repair a partial cloud by fully utilizing the scanning images and the corresponding reconstructed cloud. The repaired point cloud can achieve multi-view consistency with the captured images at high spatial resolution. We engage the input cloud to initialize the learnable implicit area and then introduce the axis-aligned triple volumes features to enhance the NeRF representation. For the finetunes of a single object, we optimize the global latent condition while retaining multi-view consistency. As a result, the generated point clouds are geometrically consistent with the partial scanning images. Extensive experiments on ShapeNet and Scannet demonstrate that our works achieve competitive performances compared to other state-of-the-art point cloud-based neural scene rendering and editing performances.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

GPN: Generative Point-Based NeRF

  • Haipeng Wang

摘要

Scanning objects with modern registration devices typically gives incomplete point clouds, primarily due to the limitations of partial scanning, 3D occlusions, and dynamic light conditions. Recent works on processing incomplete point clouds have always focused on point cloud completion. However, these approaches do not ensure consistency between the completed point cloud and the captured images regarding color and geometry. We propose using Generative Point-based NeRF (GPN) to reconstruct and repair a partial cloud by fully utilizing the scanning images and the corresponding reconstructed cloud. The repaired point cloud can achieve multi-view consistency with the captured images at high spatial resolution. We engage the input cloud to initialize the learnable implicit area and then introduce the axis-aligned triple volumes features to enhance the NeRF representation. For the finetunes of a single object, we optimize the global latent condition while retaining multi-view consistency. As a result, the generated point clouds are geometrically consistent with the partial scanning images. Extensive experiments on ShapeNet and Scannet demonstrate that our works achieve competitive performances compared to other state-of-the-art point cloud-based neural scene rendering and editing performances.