Implementation of LLM-Based Chatbots for Academic Support in Higher Education Institutions: A Case Study
摘要
This work presents an intelligent chatbot system based on Large Language Models (LLM) and Retrieval Augmented Generation (RAG) techniques specifically designed to improve access to academic information and university services at a higher education institution. The main objective is to provide accurate and contextualized responses about university resources, academic programs, administrative procedures, and student services in an interactive manner available 24/7. The system integrates university data as external knowledge corpus into RAG pipelines for domain-specific question-answering tasks in the educational context. System evaluation was conducted through quantitative metrics using the RAG (Retrieval Augmented Generation Assessment) framework, comparative analysis against baseline systems, and usability assessments through the System Usability Scale (SUS). Results demonstrate outstanding quantitative performance with an average RAG score of 0.94, statistically significant improvements over rule-based chatbots (p < 0.001), and satisfactory user experience with SUS score of 71.2. This study contributes to the educational technology field by demonstrating the viability of implementing intelligent virtual assistants in higher education institutions while establishing reproducible evaluation frameworks and addressing ethical considerations.