Knowledge Graph Question Answering for Materials Science (KGQA4MAT)

We present a study on Knowledge Graph Question Answering in Materials Science (KGQA4MAT), with a focus on metal-organic frameworks (MOFs). A knowledge graph for metal-organic frameworks (MOF-KG) has been constructed by integrating structured data, metadata, and knowledge extracted from the literature. We aim to develop a natural language (NL) interface for domain expert to query the MOF-KG. A first step is our benchmark, which consists of 161 complex questions involving comparison, aggregation, and intricate graph structures. Each question has been rephrased into three additional variations, totaling 644 questions and 161 KG queries. We then developed a systematic approach for utilizing ChatGPT to translate natural language questions into formal KG queries. We experimented with different prompt strategies. The research indicated that using an ontology, providing a few-shot examples, and offering a chain-of-thought explanation resulted in the top F1-score of 0.89. We also applied this method to the well-known QALD-9 dataset, achieving performance on par with the state-of-the-art techniques. The results indicate applicability of this model for MOF research and potentially other scientific foci.