A Multimodal Large Language Model to Support Participatory Urban Village Renovation

This study examines the potential application of multimodal large language model (MLLM) in the participatory design process of urban village renovation, with a focus on enhancing public participation and optimizing design proposals. In the context of rapid urbanization in China, urban villages face severe challenges, including aging infrastructure and overcrowding, making their revitalization a critical task in urban renewal. Current renovation projects predominantly rely on expert- and government-led decision-making, often resulting in a mismatch between design proposals and residents’ needs. This research proposes a method that integrates residents’ feedback through multimodal inputs, including images and text, to generate design solutions that align with their preferences and the local environment. The study was conducted in Dakan Village, Shenzhen, where a participatory design approach was employed to collect and analyze residents’ feedback, using GPT Vision and Latent Diffusion Model (LDM) to generate and refine design proposals. The results show that the integration of multimodal data facilitated residents’ expression of design needs, narrowing the gap between design proposals and actual needs. The study also demonstrates that MLLM technology, combined with Low-Rank Adaptation (LoRA) and ControlNet fine-tuning and prompt engineering, can generate designs that are more aligned with residents’ expectations. However, the research highlights limitations, including the need for further refinement of the designs and the challenge of model interpretability. The findings suggest that while MLLM provide significant support in participatory design, professional expertise remains essential in final design adjustments, and future studies should focus on improving model transparency and feasibility for broader applications in urban renewal projects.