Text-Driven Scene Generation
摘要
High-quality scene representations, including HDR panoramasHDR panoramas, are widely used to achieve photorealistic lightingPhotorealistic lighting and immersive reflectionsImmersive reflections in 3D graphics. However, capturing such high-fidelity scene data is often challenging, prompting the need for a versatile and user-friendly generative model that allows intuitive control. This chapter discusses a zero-shot text-driven framework for generating high-resolution 3D scenes directly from free-form textual descriptions, with no paired training data required. The approach synthesizes scenes through two key stages: (1) text-driven generation of low-resolution and low-dynamic-range representations and (2) super-resolution reconstruction to enhance both spatial resolution and dynamic range. To enable zero-shot scene generation, the approach utilizes a dual-codebook architectureDual-codebook architecture to encode diverse environmental features as discrete representations. Moreover, a global samplerGlobal sampler is designed based on a vision-language model to interpret input text and sample holistic scene semantics from the global codebook. A local sampler is then used to refine these semantics to synthesize detailed low-resolution scene representations in a patch-by-patch manner. The results not only justify the capability of generating visually compelling scenes with high fidelity but also demonstrate the practical utility of the framework in challenging scenarios such as realistic rendering and immersive virtual environments.