Understanding tourist engagement with traditional dwellings through multimodal deep learning: a case study of the Gao Family Compound, Xi’an

This study applies a multimodal deep learning approach to examine tourist engagement with traditional architectural heritage. The Gao Family Compound in Xi’an serves as a representative case. Social-media texts and images from Weibo and Xiaohongshu capture on-site experiences and expressions. Texts undergo BERT-based sentiment and topic analysis. Images are classified with a fine-tuned ViT-Large model across 31 architectural and cultural classes. The combined pipeline integrates affect, semantics, and visual attention to provide a coherent view of how tourists perceive heritage spaces. Results show a predominance of positive sentiment. Tourists focus on symbolic forms and intangible practices, including performances and crafts. Cross-modal comparison indicates strong semantic alignment between salient textual keywords and ViT-based visual classes, suggesting stable patterns of attention across media. Negative posts are few and relate mainly to service and pricing, offering direct guidance for site management. The study demonstrates how multimodal deep learning supports interpretation in heritage studies. It links the tourist gaze with affective attachment in a computational setting and clarifies which features attract attention online. The approach also informs management by identifying service frictions and by highlighting elements suited to interpretation, exhibition, and digital communication. The framework provides a basis for scalable monitoring and for future work on robustness to user-generated images.