Combining Stable Diffusion and Object Detection for Semi-automatic Environment Decoration
摘要
Enhancing immersive environments in an efficient way is essential across various industries. However, the task of selecting and placing suitable objects to decorate 3D scenes often requires a substantial investment of time and expertise. We present a novel semi-automatic approach to accelerate this process by fusing state-of-the-art generative text-to-image models with object detection. We use a rendering of a plain 3D scene as input for Stable Diffusion and generate an image depicting a decorated version. With object detection, we identify the location, size and class of decorative elements in the generated image. Based on this information, we populate the original 3D scene with matching assets from a database. Our results demonstrate the capabilities of our proposed approach in enhancing 3D scenes through the incorporation of decorative models across different scenarios. We show that our method seamlessly integrates into existing 3D content creation tools highlighting its applicability within established workflows.