Large language models (LLMs) integrated into chatbots offer new opportunities for personalized learning and student support in higher education. However, reliance on commercial LLM-based systems raises concerns around privacy, transparency, and equitable access. We present an open-source, locally deployed chatbot system designed to align with institutional values of autonomy, fairness, and academic integrity. Based on the open-source tools Open WebUI and Ollama, the system uses a model-agnostic architecture and provides features such as document upload and analysis, web search, and LDAP-based authentication. Manual evaluation by two human evaluators with overlapping expertise across seven key functional areas using 15 test cases showed a 73.3 % compliance rate, with strong performance in summarization, ethical safeguards, and prompt filtering. Limitations were found in web search reliability and gender-neutral text generation. The evaluation framework is extensible and may be used to benchmark both open-source and commercial chatbot systems. Planned future work includes formal evaluation of user adoption and usability in real educational settings.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Design and Evaluation of an Open-Source, Locally Deployed Chatbot System for Higher Education

  • Daniel Reichenpfader,
  • Denis Moser,
  • Kerstin Denecke

摘要

Large language models (LLMs) integrated into chatbots offer new opportunities for personalized learning and student support in higher education. However, reliance on commercial LLM-based systems raises concerns around privacy, transparency, and equitable access. We present an open-source, locally deployed chatbot system designed to align with institutional values of autonomy, fairness, and academic integrity. Based on the open-source tools Open WebUI and Ollama, the system uses a model-agnostic architecture and provides features such as document upload and analysis, web search, and LDAP-based authentication. Manual evaluation by two human evaluators with overlapping expertise across seven key functional areas using 15 test cases showed a 73.3 % compliance rate, with strong performance in summarization, ethical safeguards, and prompt filtering. Limitations were found in web search reliability and gender-neutral text generation. The evaluation framework is extensible and may be used to benchmark both open-source and commercial chatbot systems. Planned future work includes formal evaluation of user adoption and usability in real educational settings.