An Integrated Instant NeRF and Simulation Based 3D Reconstruction for Immersive City Twin

Interactive 3-dimensional (3D) modeling techniques play critical roles in metaverse, virtual reality, and digital twin. Neural Radiance Fields (NeRF) generate high-resolution 3D models based on many real-world image data. However, the cost is high, especially for large-scale architecture modeling scenarios. On the other hand, computer simulation can generate digital scenes with coarse-grained details, which lack accuracy and details. This paper proposes the real-sim method, that integrates NeRF and computer simulation data, such as OpenStreetMap (OSM) to generate interactive 3D architecture models for large-scale scenes. We use the Instant Neural Graphics Primitives to rebuild the key buildings which can be embedded into the simulated environment. Finally, we leverage aerial and handheld devices for fast and precise campus architecture modeling.