Scholarlink

<p>Multimodal large language models (MLLMs) offer transformative potential for architectural heritage interpretation but struggle with nuanced spatial and semantic analysis. This limitation stems from a critical shortage of large-scale, high-fidelity training data. We address this gap with a fidelity-driven data augmentation framework incorporating <i>Structural-</i> and <i>Semantic-aware Augmentation</i> modules into a diffusion model. The framework generates 1672 high-fidelity synthetic images paired with 59,884 VQA samples. Quantitative evaluations demonstrate that these synthetic images preserve spatial-structural and semantic fidelity comparable to real-world data. Furthermore, MLLMs fine-tuned on this dataset show significantly improved reasoning performance without overfitting or interference. By addressing fundamental data constraints, our framework facilitates the transition from traditional, task-specific tools to general-purpose, instruction-following MLLMs capable of supporting diverse heritage preservation and urban analysis tasks.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Fidelity-driven data augmentation for multimodal large language model on architectural heritage interpretation

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月25日

查看全文
  • Rong Huang,
  • Hai-Chuan Lin,
  • Wei Zeng

摘要

Multimodal large language models (MLLMs) offer transformative potential for architectural heritage interpretation but struggle with nuanced spatial and semantic analysis. This limitation stems from a critical shortage of large-scale, high-fidelity training data. We address this gap with a fidelity-driven data augmentation framework incorporating Structural- and Semantic-aware Augmentation modules into a diffusion model. The framework generates 1672 high-fidelity synthetic images paired with 59,884 VQA samples. Quantitative evaluations demonstrate that these synthetic images preserve spatial-structural and semantic fidelity comparable to real-world data. Furthermore, MLLMs fine-tuned on this dataset show significantly improved reasoning performance without overfitting or interference. By addressing fundamental data constraints, our framework facilitates the transition from traditional, task-specific tools to general-purpose, instruction-following MLLMs capable of supporting diverse heritage preservation and urban analysis tasks.