Scalable Knowledge Graph Construction from Unstructured Text: A Case Study on Artisanal and Small-Scale Gold Mining
摘要
Constructing knowledge graphs (KGs) from unstructured text is an important yet challenging task, often requiring a great deal of manual effort and domain-specific adjustments. We propose an unsupervised, scalable method that automates both triplet extraction (subject, predicate, object) and validation using large language models (LLMs). Our framework leverages PageRank to rank external knowledge sources from the web, which are provided as contextual input to multiple LLM queries. Majority voting is then performed in the output space of these queries to ensure relationship validation. This approach offers a generalizable, domain-agnostic solution for KG construction across diverse fields. As a case study, we applied the method to Artisanal and Small-Scale Gold Mining (ASGM), constructing a knowledge graph from 1,899 triplets extracted from 9 domain-specific documents, cumulatively amounting to approximately 930 pages of unstructured text. Our framework achieves comparable performance with five baselines on a publicly available KG benchmark, and achieves over \(90\%\) accuracy on ASGM-KG, as validated by domain experts. Additionally, our framework was able to identify factual inaccuracies in popular benchmarks like Codex, highlighting the need for more reliable validation methods.