Scholarlink

<p>Existing Artificial Intelligence Generated Content (AIGC) methods often directly generate 3D geometry end-to-end from latent variables or text embeddings when processing natural language instructions, skipping the explicit planning stage of spatial layout. This results in the generated object positions, orientations, and scales not matching the semantic descriptions. To address the semantic-geometric gap and insufficient structural controllability issues of AIGC in virtual reality scene generation, this paper proposes an end-to-end framework that integrates multi-granularity semantic alignment and hierarchical constraint generation. First, a large-scale visual-language model fine-tuned from a 3D scene corpus is used to parse the user’s input natural language description, extracting object-level semantic labels, spatial relationships, and style attributes. Then, a learnable semantic-layout mapping module is used to transform high-level semantics into a 2.5D sketch with topological constraints. Based on this, a scene generator based on Neural Radiance Fields (NeRF) and diffusion prior joint optimization is constructed, applying geometric consistency loss and semantic segmentation supervision to ensure the dual fidelity of the generated 3D content in terms of structural rationality and texture details. The proposed method was compared and validated on the 3D-FRONT dataset with four baseline methods, including PROX and Scene Generative Adversarial Network (SceneGAN). Experiments show that the proposed method achieves a semantic matching accuracy of 78.4 ± 1.9%, a layout FID reduced to 21.3 ± 1.5, a user controllability score of 4.32 ± 0.47/5, a physical plausibility score of 92.3 ± 1.7, and a geometric IoU improved to 63.8 ± 1.8%, significantly outperforming other baseline models. This research confirms that by explicitly modeling the hierarchical mapping relationship between semantics, layout, and geometry, the interpretability, controllability, and engineering usability of Virtual Reality (VR) scene generation can be effectively improved, providing a systematic solution for the intelligent creation of high-fidelity VR content.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Semantic understanding and controllable generation methods of AIGC in virtual reality scene construction

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月12日

查看全文
  • Jiaji Zhou,
  • Jiabai Jin,
  • Peng Zhuang,
  • Dongdong Jia,
  • Chunli Wu

摘要

Existing Artificial Intelligence Generated Content (AIGC) methods often directly generate 3D geometry end-to-end from latent variables or text embeddings when processing natural language instructions, skipping the explicit planning stage of spatial layout. This results in the generated object positions, orientations, and scales not matching the semantic descriptions. To address the semantic-geometric gap and insufficient structural controllability issues of AIGC in virtual reality scene generation, this paper proposes an end-to-end framework that integrates multi-granularity semantic alignment and hierarchical constraint generation. First, a large-scale visual-language model fine-tuned from a 3D scene corpus is used to parse the user’s input natural language description, extracting object-level semantic labels, spatial relationships, and style attributes. Then, a learnable semantic-layout mapping module is used to transform high-level semantics into a 2.5D sketch with topological constraints. Based on this, a scene generator based on Neural Radiance Fields (NeRF) and diffusion prior joint optimization is constructed, applying geometric consistency loss and semantic segmentation supervision to ensure the dual fidelity of the generated 3D content in terms of structural rationality and texture details. The proposed method was compared and validated on the 3D-FRONT dataset with four baseline methods, including PROX and Scene Generative Adversarial Network (SceneGAN). Experiments show that the proposed method achieves a semantic matching accuracy of 78.4 ± 1.9%, a layout FID reduced to 21.3 ± 1.5, a user controllability score of 4.32 ± 0.47/5, a physical plausibility score of 92.3 ± 1.7, and a geometric IoU improved to 63.8 ± 1.8%, significantly outperforming other baseline models. This research confirms that by explicitly modeling the hierarchical mapping relationship between semantics, layout, and geometry, the interpretability, controllability, and engineering usability of Virtual Reality (VR) scene generation can be effectively improved, providing a systematic solution for the intelligent creation of high-fidelity VR content.