<p>Indoor scene data play a crucial role across various domains, particularly with the rapid advancement of large language models (LLMs), where embodied intelligence is a key research focus. However, existing indoor scene datasets are often limited in quantity and lack comprehension of depth information, resulting in generated scenes that frequently appear inauthentic and incoherent. To address these limitations, we propose a language-guided method for generating complex indoor scenes, enabling users to create detailed environments from diverse inputs. Our approach begins by extracting the fundamental structural framework and spatial relationships of a house from user-provided text descriptions, ensuring the authenticity of the generated scenes. Subsequently, LLM agents generate a scene graph that outlines objects and their interrelations. This graph is refined using algorithms that optimize object placement and orientation, enhancing the overall coherence of the layout. Finally, we retrieve assets from an existing database and orient them appropriately. Extensive qualitative and quantitative experiments demonstrate that our method excels in producing high-quality, diverse, and realistic complex indoor scenes, maintaining strong consistency with user inputs. Moreover, preliminary validation of our method has shown that the generated indoor scene data contain detailed annotation and geometric information. This richness is expected to enhance the perceptual and cognitive capabilities of embodied intelligence in the 3D physical world.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Text2Scenes: Language-Guided Synthesis of Complex Indoor Scenes

  • Haowei Liu,
  • Xintong Dong,
  • Chuanyang Li,
  • Zhouwang Yang,
  • Yanzhi Song

摘要

Indoor scene data play a crucial role across various domains, particularly with the rapid advancement of large language models (LLMs), where embodied intelligence is a key research focus. However, existing indoor scene datasets are often limited in quantity and lack comprehension of depth information, resulting in generated scenes that frequently appear inauthentic and incoherent. To address these limitations, we propose a language-guided method for generating complex indoor scenes, enabling users to create detailed environments from diverse inputs. Our approach begins by extracting the fundamental structural framework and spatial relationships of a house from user-provided text descriptions, ensuring the authenticity of the generated scenes. Subsequently, LLM agents generate a scene graph that outlines objects and their interrelations. This graph is refined using algorithms that optimize object placement and orientation, enhancing the overall coherence of the layout. Finally, we retrieve assets from an existing database and orient them appropriately. Extensive qualitative and quantitative experiments demonstrate that our method excels in producing high-quality, diverse, and realistic complex indoor scenes, maintaining strong consistency with user inputs. Moreover, preliminary validation of our method has shown that the generated indoor scene data contain detailed annotation and geometric information. This richness is expected to enhance the perceptual and cognitive capabilities of embodied intelligence in the 3D physical world.