Enhancing Large Language Models Through External Domain Knowledge

Large Language Models (LLM) demonstrate promising results in generating content with current fine-tuning and prompting methods. Yet, they have limited application in industrial knowledge management or specific expert domains, due to weak factuality and safety-critical hallucination. Therefore, it is necessary to enhance the language model with external knowledge. The provision and representation of the external knowledge holds several challenges and problems. This paper proposes a human-centred LLM-based system architecture designed as a modular extension, which improves the overall factuality of the generated output. Following a design science research approach, first the problems and objectives of the research are identified. In the next step the artifact is developed based on requirements deducted from literature. Eventually, the functionality of the artifact is demonstrated as a proof-of-concept in a case study. The research contributes an initial approach for effective and grounded knowledge transfer, which minimizes the risk of hallucination from LLM-generated content.