Semantic Exploration of Industrial Standards Using Large Language Models

This paper investigates the effectiveness of using Large Language Models (LLMs) within specialized domains, such as industrial standards semantic exploration. While pre-trained LLMs have demonstrated remarkable proficiency in various generic language tasks, their application to domain-specific knowledge tasks requires a cautious approach due to the potential risk of hallucination in responses. Recently, the Retrieval Augmented Generation (RAG) framework has been proposed to enhance pre-trained LLMs with data retrieval mechanisms, enabling them to incorporate precise, context-relevant information from external data sources. Motivated by this, we adapted a RAG approach to the context of the OPC UA industrial standard documentation. To ensure a fair evaluation, we introduced a preliminary benchmark dataset comprising the most prevalent questions and answers related to the OPC UA standard. Our experimental results indicate that the RAG approach can be effectively employed for the semantic search of OPC UA standard documentation. We found the RAG approach significantly improves faithfulness of existing pre-trained LLMs, offering substantial potential for further exploration and generalization towards a comprehensive semantic understanding approach for industrial standards on a global scale.