Scholarlink

Retrieval Augmented Generation (RAG) has been pivotal in the utilization of Large Language Models (LLM) to improve the factuality of long-form question answering systems. Knowledge graphs (KG) represent a linking of disparate sources that potentially yield useful information for mitigating the issues of insufficient knowledge and hallucination within the LLM-RAG pipeline. However, the creation of domain-specific KG is costly and usually requires domain experts. To alleviate the above challenges, this work proposes LARK, a novel domain-specific question answering framework to enhance the knowledge capabilities of LLM by integrating structured KG, thereby significantly reducing the reliance on the “generic” latent knowledge of LLMs. Here, we showcase how LLMs can be deployed to not only act in dynamic information retrieval and in answer generating frameworks, but also as a flexible agents to automatically extract relevant entities and relations for automated construction of domain-specific KGs. Crucially, our proposed pairing of question decomposition and semantic triplet fact retrieval helps in optimal subgraph retrieval methodology for RAG. Experimental evaluations on financial domain public dataset, verified with human evaluations, demonstrate that our framework enables a robust pipeline incorporating schema-free KG within a RAG framework to improve QA accuracy by nearly 13% on FinanceBench data.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

LARK: Integrating LLM-Based KG Construction and RAG for Financial Question Answering

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Edward Burgin,
  • Sourav Dutta,
  • Mingxue Wang

摘要

Retrieval Augmented Generation (RAG) has been pivotal in the utilization of Large Language Models (LLM) to improve the factuality of long-form question answering systems. Knowledge graphs (KG) represent a linking of disparate sources that potentially yield useful information for mitigating the issues of insufficient knowledge and hallucination within the LLM-RAG pipeline. However, the creation of domain-specific KG is costly and usually requires domain experts. To alleviate the above challenges, this work proposes LARK, a novel domain-specific question answering framework to enhance the knowledge capabilities of LLM by integrating structured KG, thereby significantly reducing the reliance on the “generic” latent knowledge of LLMs. Here, we showcase how LLMs can be deployed to not only act in dynamic information retrieval and in answer generating frameworks, but also as a flexible agents to automatically extract relevant entities and relations for automated construction of domain-specific KGs. Crucially, our proposed pairing of question decomposition and semantic triplet fact retrieval helps in optimal subgraph retrieval methodology for RAG. Experimental evaluations on financial domain public dataset, verified with human evaluations, demonstrate that our framework enables a robust pipeline incorporating schema-free KG within a RAG framework to improve QA accuracy by nearly 13% on FinanceBench data.