Retrieval-Augmented Generation (RAG) has proven effective in educational contexts for improving the factual grounding of large language models (LLMs), yet most systems rely solely on unstructured text retrieval. While recent approaches introduce knowledge graphs for better structure, they often fail to address complex, multimodal queries essential to effective instruction. In response, we propose a graph-enhanced RAG system that integrates text and image-derived instructional materials—such as lecture slides and diagrams—into a unified knowledge graph. The system employs a dual-stage pipeline: first retrieving content via semantic and graph-based search, then summarizing it into prompt-efficient inputs for generation. This design supports pedagogically aligned, explainable answers and enhances transparency through source-level similarity tracing. We validate our system using a dataset derived from lecture slides in a Master’s-level deep learning course, comparing three configurations: a baseline text-only retriever, a dual-stage summarizer, and a full multimodal variant with dual-stage summarization using both text and image inputs. Results show that combining multimodal retrieval with graph-based summarization produces more coherent and context-rich responses, particularly for visually grounded or conceptually layered questions.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Graph Summarization for Educational RAG: Two-Stage Retrieval with Structured and Multimodal Inputs

  • Suronapee Phoomvuthisarn

摘要

Retrieval-Augmented Generation (RAG) has proven effective in educational contexts for improving the factual grounding of large language models (LLMs), yet most systems rely solely on unstructured text retrieval. While recent approaches introduce knowledge graphs for better structure, they often fail to address complex, multimodal queries essential to effective instruction. In response, we propose a graph-enhanced RAG system that integrates text and image-derived instructional materials—such as lecture slides and diagrams—into a unified knowledge graph. The system employs a dual-stage pipeline: first retrieving content via semantic and graph-based search, then summarizing it into prompt-efficient inputs for generation. This design supports pedagogically aligned, explainable answers and enhances transparency through source-level similarity tracing. We validate our system using a dataset derived from lecture slides in a Master’s-level deep learning course, comparing three configurations: a baseline text-only retriever, a dual-stage summarizer, and a full multimodal variant with dual-stage summarization using both text and image inputs. Results show that combining multimodal retrieval with graph-based summarization produces more coherent and context-rich responses, particularly for visually grounded or conceptually layered questions.