<p>Novel view synthesis of dynamic 3D scenes is an attractive but challenging problem. Many recent methods extend the 3D Gaussian Splatting model with temporal attributes in order to achieve high-quality and real-time rendered dynamic 3D scene representation. In order to have a more complete understanding of 4D scenes, we propose Spacetime Gaussian Grouping (SGG), for segmenting and tracking the objects spatially and temporally. The proposed method trains the Spacetime Gaussians (STG) model in conjunction with the multi-view consistent segmentation masks corresponding to the input images, which are used to label the 3D Gaussians with instance identities. Our model produces a lightweight, dynamic 3D representation of a scene, enabling users to select and edit specific objects in a 4D manner across various devices, including smartphones, PCs, VR systems, among other devices. Furthermore, a novel dynamic view synthesis segmentation benchmark is proposed to evaluate our method quantitatively.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SGG: Spacetime Gaussian Grouping for 4D object segmentation

  • Bangning Wei,
  • Joshua Maraval,
  • Baptiste Besnard,
  • Meriem Outtas,
  • Kidiyo Kpalma,
  • Nicolas Ramin,
  • Lu Zhang

摘要

Novel view synthesis of dynamic 3D scenes is an attractive but challenging problem. Many recent methods extend the 3D Gaussian Splatting model with temporal attributes in order to achieve high-quality and real-time rendered dynamic 3D scene representation. In order to have a more complete understanding of 4D scenes, we propose Spacetime Gaussian Grouping (SGG), for segmenting and tracking the objects spatially and temporally. The proposed method trains the Spacetime Gaussians (STG) model in conjunction with the multi-view consistent segmentation masks corresponding to the input images, which are used to label the 3D Gaussians with instance identities. Our model produces a lightweight, dynamic 3D representation of a scene, enabling users to select and edit specific objects in a 4D manner across various devices, including smartphones, PCs, VR systems, among other devices. Furthermore, a novel dynamic view synthesis segmentation benchmark is proposed to evaluate our method quantitatively.