Constructing knowledge graphs (KGs) from unstructured text is an important yet challenging task, often requiring a great deal of manual effort and domain-specific adjustments. We propose an unsupervised, scalable method that automates both triplet extraction (subject, predicate, object) and validation using large language models (LLMs). Our framework leverages PageRank to rank external knowledge sources from the web, which are provided as contextual input to multiple LLM queries. Majority voting is then performed in the output space of these queries to ensure relationship validation. This approach offers a generalizable, domain-agnostic solution for KG construction across diverse fields. As a case study, we applied the method to Artisanal and Small-Scale Gold Mining (ASGM), constructing a knowledge graph from 1,899 triplets extracted from 9 domain-specific documents, cumulatively amounting to approximately 930 pages of unstructured text. Our framework achieves comparable performance with five baselines on a publicly available KG benchmark, and achieves over \(90\%\) accuracy on ASGM-KG, as validated by domain experts. Additionally, our framework was able to identify factual inaccuracies in popular benchmarks like Codex, highlighting the need for more reliable validation methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Scalable Knowledge Graph Construction from Unstructured Text: A Case Study on Artisanal and Small-Scale Gold Mining

  • Debashis Gupta,
  • Aditi Golder,
  • Sahil Sidheekh,
  • Sakib Imtiaz,
  • Sarra Alaqahtani,
  • Fan Yang,
  • Greg Larsen,
  • Miles Silman,
  • Luis Fernendez,
  • Robert Plemmons,
  • Sriraam Natarajan,
  • V. Paul Pauca

摘要

Constructing knowledge graphs (KGs) from unstructured text is an important yet challenging task, often requiring a great deal of manual effort and domain-specific adjustments. We propose an unsupervised, scalable method that automates both triplet extraction (subject, predicate, object) and validation using large language models (LLMs). Our framework leverages PageRank to rank external knowledge sources from the web, which are provided as contextual input to multiple LLM queries. Majority voting is then performed in the output space of these queries to ensure relationship validation. This approach offers a generalizable, domain-agnostic solution for KG construction across diverse fields. As a case study, we applied the method to Artisanal and Small-Scale Gold Mining (ASGM), constructing a knowledge graph from 1,899 triplets extracted from 9 domain-specific documents, cumulatively amounting to approximately 930 pages of unstructured text. Our framework achieves comparable performance with five baselines on a publicly available KG benchmark, and achieves over \(90\%\) accuracy on ASGM-KG, as validated by domain experts. Additionally, our framework was able to identify factual inaccuracies in popular benchmarks like Codex, highlighting the need for more reliable validation methods.