Enhancing immersive environments in an efficient way is essential across various industries. However, the task of selecting and placing suitable objects to decorate 3D scenes often requires a substantial investment of time and expertise. We present a novel semi-automatic approach to accelerate this process by fusing state-of-the-art generative text-to-image models with object detection. We use a rendering of a plain 3D scene as input for Stable Diffusion and generate an image depicting a decorated version. With object detection, we identify the location, size and class of decorative elements in the generated image. Based on this information, we populate the original 3D scene with matching assets from a database. Our results demonstrate the capabilities of our proposed approach in enhancing 3D scenes through the incorporation of decorative models across different scenarios. We show that our method seamlessly integrates into existing 3D content creation tools highlighting its applicability within established workflows.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Combining Stable Diffusion and Object Detection for Semi-automatic Environment Decoration

  • Philipp Drescher,
  • Irena Ruprecht,
  • Volker Settgast

摘要

Enhancing immersive environments in an efficient way is essential across various industries. However, the task of selecting and placing suitable objects to decorate 3D scenes often requires a substantial investment of time and expertise. We present a novel semi-automatic approach to accelerate this process by fusing state-of-the-art generative text-to-image models with object detection. We use a rendering of a plain 3D scene as input for Stable Diffusion and generate an image depicting a decorated version. With object detection, we identify the location, size and class of decorative elements in the generated image. Based on this information, we populate the original 3D scene with matching assets from a database. Our results demonstrate the capabilities of our proposed approach in enhancing 3D scenes through the incorporation of decorative models across different scenarios. We show that our method seamlessly integrates into existing 3D content creation tools highlighting its applicability within established workflows.