<p>Single-view 3D reconstruction often struggles with semantically controllable editing due to the lack of high-level semantic guidance. To address this, we introduce LSVR-SE, a language-guided framework for single-view reconstruction with semantic editing. Our framework integrates a Hierarchical Semantic Disentanglement Encoder (HSDE) for precise 3D cross-modal alignment, a Language-Conditioned Neural Radiance Field (LC-NeRF) for real-time, text-driven modulation of the radiance field, and a Differentiable Programmatic Editing Engine (DPEE) for parsing language into differentiable operations while enforcing topological and physical constraints. Experiments on DTU, Objaverse-LVIS, and a custom BIM dataset demonstrate that LSVR-SE outperforms state-of-the-art methods, achieving a 3.4&#xa0;dB improvement in PSNR and a 17.8% increase in editing IoU. Notably, our framework pioneers zero-shot text-to-BIM conversion, enabling the generation of compliant IFC files from natural language. Code is open-sourced at <a href="https://github.com/LLxuLL/LSVR-SE">https://github.com/LLxuLL/LSVR-SE</a>.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Language-guided semantic editing in single-view 3D reconstruction

  • Maoyang Xu,
  • Guanglei Qi,
  • Nana He,
  • Fujiao Ju

摘要

Single-view 3D reconstruction often struggles with semantically controllable editing due to the lack of high-level semantic guidance. To address this, we introduce LSVR-SE, a language-guided framework for single-view reconstruction with semantic editing. Our framework integrates a Hierarchical Semantic Disentanglement Encoder (HSDE) for precise 3D cross-modal alignment, a Language-Conditioned Neural Radiance Field (LC-NeRF) for real-time, text-driven modulation of the radiance field, and a Differentiable Programmatic Editing Engine (DPEE) for parsing language into differentiable operations while enforcing topological and physical constraints. Experiments on DTU, Objaverse-LVIS, and a custom BIM dataset demonstrate that LSVR-SE outperforms state-of-the-art methods, achieving a 3.4 dB improvement in PSNR and a 17.8% increase in editing IoU. Notably, our framework pioneers zero-shot text-to-BIM conversion, enabling the generation of compliant IFC files from natural language. Code is open-sourced at https://github.com/LLxuLL/LSVR-SE.