A Multimodal Retrieval-Augmented Generation System for Intelligent Question Answering
摘要
With the rapid advancement of multimodal large models, there is an increasing demand for intelligent question-answering systems capable of comprehending and leveraging complex, heterogeneous, and multi-source knowledge. However, existing approaches often face challenges in semantic alignment, efficient retrieval, and coherent generation, particularly when processing unstructured multimodal data and multi-turn dialogues. To address these limitations, this study presents a systematically designed multimodal retrieval-augmented generation (RAG) system that integrates advanced knowledge segmentation, hybrid retrieval, and generation techniques. In the offline stage, the system performs fine-grained knowledge segmentation and structured annotation through multimodal understanding, while enhancing global retrieval performance via multi-dimensional knowledge augmentation. In the online stage, it establishes an end-to-end, closed-loop answering pipeline that incorporates multi-turn dialogue rewriting, hybrid retrieval, reranking, and large model generation. Furthermore, a comprehensive automatic evaluation framework is developed to assess the system across multiple dimensions, including retrieval accuracy, factual consistency, generation quality, and structural clarity. Experimental results show that the proposed system substantially improves retrieval hit rates and generation performance over baseline methods. Overall, this work demonstrates the effectiveness of integrating RAG frameworks with multimodal large models and provides a solid foundation for deployment in high-demand domains such as healthcare, finance, and education.