We consider a special case of knowledge graph construction from text, where the target knowledge graph is structured as specific Directed Acyclic Graph and the input text has the form of a recipe. The intention of this paper is to present a case study that uses a large language model (LLM) for the knowledge extraction process. We formulate knowledge extraction as a model-driven structure recovery process and demonstrate that LLMs can be effectively used in the process. We demonstrate through extensive experiments that using LLMs a zero-shot process produces a wide range of errors. To remedy them, we propose two different model-driven prompting strategies by which LLMs can be used to improve the accuracy of knowledge graph construction. We demonstrate that a state memoization technique introduces an accuracy-efficiency tradeoff that demands further research.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Building Model-Driven Knowledge Graphs via Large Language Models

  • Vaaruni Desai,
  • Yinglan Chi,
  • Jon Stephens,
  • Amarnath Gupta

摘要

We consider a special case of knowledge graph construction from text, where the target knowledge graph is structured as specific Directed Acyclic Graph and the input text has the form of a recipe. The intention of this paper is to present a case study that uses a large language model (LLM) for the knowledge extraction process. We formulate knowledge extraction as a model-driven structure recovery process and demonstrate that LLMs can be effectively used in the process. We demonstrate through extensive experiments that using LLMs a zero-shot process produces a wide range of errors. To remedy them, we propose two different model-driven prompting strategies by which LLMs can be used to improve the accuracy of knowledge graph construction. We demonstrate that a state memoization technique introduces an accuracy-efficiency tradeoff that demands further research.