Archi-Nerf: View Synthesis for Traditional Chinese Architecture Using Nerf Neural Network
摘要
Traditional Neural Radiance Fields (NeRF) models are designed to generate entire scenes, which limits their applicability in architectural design. Architectural workflows often require component-based models or detailed visual representations to focus on specific parts of a building, a need rarely addressed by generative networks. Furthermore, architects conventionally rely on manual methods or rule-based tools such as Grasshopper (GH), resulting in minimal integration between generative AI and architectural modeling workflows. This study proposes a pipeline for generating 3D representations of specific architectural elements using video data collected from drones or online sources. The Hanging Temple (Xuankong Temple) serves as a case study to demonstrate the pipeline. The process begins with extracting frames from videos to obtain multi-angle images of the temple. These images are segmented using the Grounded Segment Anything Model (SAM) and masked based on user-defined requirements, such as isolating the whole scene, natural elements, architectural structures, or specific components. The filtered image dataset is then processed in COLMAP to compute spatial coordinates (x,y, z) and viewing directions (θ,ϕ). This information, along with the image dataset, is fed into Instant-NGP, a refined NeRF framework, to generate 3D representations. The pipeline successfully generates independent 3D models of the entire Hanging Temple, the natural cliff, the temple structure, and its roof. Although the solid meshes derived from these 3D representations may lack full precision, the results demonstrate the potential of AI to create detailed 3D models of individual architectural elements. This approach reduces manual effort, supports iterative design workflows, and facilitates the development of complex architectural models, contributing to advancements in computational design and architectural analysis.