Large language models (LLMs) create a new paradigm for Document-level Relation Extraction (DocRE). Despite their advancements, existing LLM-based methods still face challenges, including inadequate utilization of reasoning capabilities and potential omission of intricate implicit relation due to manually designed joint extraction processes, leading to suboptimal performance. In this paper, we propose ToolRE, a novel framework that leverages LLMs for DocRE through self-verification and tool-assisted relational reasoning. First, we design three tools for entity extraction, explicit relation extraction, and complex relation extraction. We progressively refine the extraction results by allowing the agentic user to select tools. We adopt a tuning-free strategy that utilizes LLMs to reason over trigger words and relation through reasoning. Additionally, we implement backward verification to obtain interpretable relation verification scores, thereby enhancing the performance of relation extraction. Experimental results on the DocRE benchmark demonstrate that our method significantly outperforms baseline LLM-based DocRE models.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Leveraging Large Language Models for Document-Level Complex Relation Extraction Through Self-Verification and Tool-Assisted Relational Reasoning

  • Yu Wang,
  • Yipu Chen,
  • Zhen Wang,
  • Liqiang Wen,
  • Wen Zhao

摘要

Large language models (LLMs) create a new paradigm for Document-level Relation Extraction (DocRE). Despite their advancements, existing LLM-based methods still face challenges, including inadequate utilization of reasoning capabilities and potential omission of intricate implicit relation due to manually designed joint extraction processes, leading to suboptimal performance. In this paper, we propose ToolRE, a novel framework that leverages LLMs for DocRE through self-verification and tool-assisted relational reasoning. First, we design three tools for entity extraction, explicit relation extraction, and complex relation extraction. We progressively refine the extraction results by allowing the agentic user to select tools. We adopt a tuning-free strategy that utilizes LLMs to reason over trigger words and relation through reasoning. Additionally, we implement backward verification to obtain interpretable relation verification scores, thereby enhancing the performance of relation extraction. Experimental results on the DocRE benchmark demonstrate that our method significantly outperforms baseline LLM-based DocRE models.