GauScene: Physically Plausible Scene Generation via Language-Guided 3D Gaussian Interaction
摘要
Generating physically plausible and controllable 3D dynamic scenes remains a central challenge in computer graphics and vision. While existing methods show promise, they often struggle to reconcile the demands of realistic physics simulation with the need for intuitive, user-driven control. We present a novel framework that synergizes generative AI with robust physics-based simulation, all grounded in a highly efficient 3D Gaussian representation. Our method employs a multimodal pipeline that integrates state-of-the-art models for both foreground asset creation and background scene understanding. A core innovation is a Large Language Model (LLM) agent that functions as a semantic actuator, translating natural language commands into physical forces to enable intuitive, real-time control over scene dynamics. To ensure stability and physical realism, we introduce a novel simulation engine featuring spatiotemporally consistent grid decoupling to preserve static background integrity and a high-fidelity collision model based on Signed Distance Fields (SDFs). Comprehensive evaluations on benchmarks like ScanNet demonstrate state-of-the-art performance, achieving a rendering quality of 28.7 PSNR while operating at 17.3 FPS. Experiments confirm that our framework provides a practical and powerful solution for interactive dynamic scene generation.