Evidence-Based Design of a Knowledge Chatbot: From Interaction Mechanisms to Design Principles
摘要
This paper discusses how internal chatbots can support enterprise knowledge work by reducing the time and effort needed for knowledge retrieval. Although effective knowledge management is essential for organizational success, employees often spend a significant amount of time searching for scattered information and clarifying unclear documentation. Recent advances in large language models and retrieval-augmented generation allow users to access knowledge sources through conversation. However, many enterprises struggle with trust, scope clarity, and day-to-day adoption. To investigate the design of a feasible and useful knowledge chatbot, we employed a design science research approach in the finance department of a medium-sized German company. Based on a focused literature review and three expert interviews, we derived 14 design requirements and consolidated them into six design principles: in-flow integration, transparency, guided access, clear scope and handover, privacy by design, and a clean data foundation with learning loops. These principles inform the development of a Teams-integrated knowledge chatbot built with Microsoft Copilot Studio. We evaluated the artifact through a formative demonstration with seven experts and a summative field study with fifteen employees. The results show high perceived usability and promising support in task completion but also reveal issues in navigation and content coverage. This paper provides empirically grounded design principles and actionable guidance for implementing LLM-based knowledge chatbots in enterprise settings.