Scholarlink

<p>Accurate 3D instance segmentation is foundational for perception and decision making in embodied systems, yet prevailing approaches depend on densely annotated 3D point clouds, which are costly to acquire and difficult to scale across sensors and environments. We address this bottleneck with <i>Fusion3DGS</i>, an end-to-end, label efficient framework that couples 3D Gaussian Splatting with coordinated 2D–3D neural processing. From multi-view RGB images equipped only with 2D instance masks, our method optimizes a compact anisotropic Gaussian scene representation and performs instance reasoning via an occlusion aware cross-attention fusion stack. The weight-sharing lock imposes shape-consistent, gated coupling of early 2D and 3D kernels with conservative feedback and drift regularization to stabilize 2D-mask training and improve label efficiency, and a rendering consistency objective ties the Gaussian geometry to 2D supervision, enhancing boundary fidelity under occlusion and view changes. The ability to learn from widely available RGB data without dense 3D labels makes the approach practical for large-scale deployment.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

End-to-end Fusion3DGS: label-efficient multi-modal 3D instance segmentation based on Gaussian splatting

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年1月5日

查看全文
  • Zhiyuan Wang,
  • Chuang Luo,
  • Jianping Zhang,
  • Junhui Li,
  • Yunxiang Chen,
  • Guangyuan Zhang

摘要

Accurate 3D instance segmentation is foundational for perception and decision making in embodied systems, yet prevailing approaches depend on densely annotated 3D point clouds, which are costly to acquire and difficult to scale across sensors and environments. We address this bottleneck with Fusion3DGS, an end-to-end, label efficient framework that couples 3D Gaussian Splatting with coordinated 2D–3D neural processing. From multi-view RGB images equipped only with 2D instance masks, our method optimizes a compact anisotropic Gaussian scene representation and performs instance reasoning via an occlusion aware cross-attention fusion stack. The weight-sharing lock imposes shape-consistent, gated coupling of early 2D and 3D kernels with conservative feedback and drift regularization to stabilize 2D-mask training and improve label efficiency, and a rendering consistency objective ties the Gaussian geometry to 2D supervision, enhancing boundary fidelity under occlusion and view changes. The ability to learn from widely available RGB data without dense 3D labels makes the approach practical for large-scale deployment.