LECTRA: Bridging Video Lectures and Student Queries via a RAG-Based Chatbot for Timestamped and Semantic Responses

In higher education, recorded lecture videos are indispensable resources, yet students often face challenges when trying to extract specific information from these lengthy materials. Whether due to complex subject matter, fast-paced delivery, or insufficient navigation tools, learners frequently struggle to revisit precise moments that would aid their understanding. This results in increased cognitive load, reduced study efficiency, and declining academic confidence. Existing solutions such as keyword-based search or playback controls lack contextual awareness, provide no timestamp linkage, and are ineffective in addressing student queries with semantic precision. Furthermore, current AI-based educational tools predominantly focus on short-form content and overlook the needs of long-form academic lectures. To address these limitations, we introduce LECTRA, a Scene-Aware Retrieval-Augmented Generation (RAG) chatbot framework designed for timestamped question answering from lecture videos. LECTRA integrates automatic speech recognition (ASR), optical character recognition (OCR), and scene detection to extract multimodal information from video lectures. The system embeds this content into a vector database and, upon receiving a student query, performs semantic similarity search to retrieve relevant segments. A large language model (LLM) then generates an answer grounded in the retrieved scene, complete with timestamp and scene reference. We evaluate LECTRA using both time-based metrics and semantic-based metrics. Experimental results on long-form lecture videos show that GPT-4o achieves the highest balance between semantic and temporal accuracy, while Gemini Flash 2.0 demonstrates the greatest faithfulness to source content. Compared to manual review, LECTRA significantly reduces information retrieval time and improves students’ ability to learn independently. By combining multimodal content extraction with generative AI and timestamped grounding, LECTRA advances the state of AI-enhanced educational tools, offering a scalable and interpretable solution that meaningfully supports student learning from video lectures.