Large Language Models (LLMs) are powerful tools for various Natural Language Processing (NLP) tasks. However, they remain prone to hallucinations and faithfulness errors, particularly in safety-critical areas such as aircraft maintenance, where errors could have severe consequences. Retrieval-Augmented Generation (RAG) approaches mitigate this issue by grounding answers in external documentation. However, these methods do not prevent inconsistencies between the retrieved context and the generated answer. These errors are referred to as context inconsistencies and can lead to unreliable outputs. In the context of aircraft maintenance, ensuring faithful answers to the context is crucial to provide correct procedures to be followed by maintenance operators. This research proposal focuses on Speculative RAG, a paradigm where the language model dynamically switches between free generation and verbatim retrieval from relevant external documentation. Recent works demonstrate that Speculative RAG can output hybrid answers, which are partially generated and partially extracted from the context, thus improving factuality, faithfulness, and transparency. The objective of this thesis is to develop and assess a framework of hybrid text generation, specifically adapted to address the challenges related to technical documentation in aircraft maintenance. Expected contributions include (1) methods for dynamic retrieval-generation control, (2) metrics for faithfulness, accuracy and fluency in this hybrid generation, and (3) validation on real aircraft maintenance documentation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Speculative Retrieval-Augmented Generation for Faithful Question Answering in Aircraft Maintenance

  • Quentin Signé

摘要

Large Language Models (LLMs) are powerful tools for various Natural Language Processing (NLP) tasks. However, they remain prone to hallucinations and faithfulness errors, particularly in safety-critical areas such as aircraft maintenance, where errors could have severe consequences. Retrieval-Augmented Generation (RAG) approaches mitigate this issue by grounding answers in external documentation. However, these methods do not prevent inconsistencies between the retrieved context and the generated answer. These errors are referred to as context inconsistencies and can lead to unreliable outputs. In the context of aircraft maintenance, ensuring faithful answers to the context is crucial to provide correct procedures to be followed by maintenance operators. This research proposal focuses on Speculative RAG, a paradigm where the language model dynamically switches between free generation and verbatim retrieval from relevant external documentation. Recent works demonstrate that Speculative RAG can output hybrid answers, which are partially generated and partially extracted from the context, thus improving factuality, faithfulness, and transparency. The objective of this thesis is to develop and assess a framework of hybrid text generation, specifically adapted to address the challenges related to technical documentation in aircraft maintenance. Expected contributions include (1) methods for dynamic retrieval-generation control, (2) metrics for faithfulness, accuracy and fluency in this hybrid generation, and (3) validation on real aircraft maintenance documentation.