Dynamic Reasoning with Language Model and Knowledge Graph for Question Answering

The question answering(QA) involves reasoning about the context and latent knowledge of complex textual descriptions. Current research is how to effectively utilize knowledge graph(KG) to enhance language model(LM) with external knowledge. In previous works, the interactions between the QA context and KG were limited, and KG input to the model contained noisy nodes, greatly restricting the model’s reasoning ability. We propose a dynamic reasoning model, DLM-KG, which is based on LM and KG. It resolves the above challenges through dynamic hierarchical interaction between QA context and KG, joint reasoning between LM and KG, and dynamic pruning of the KG. Specifically, DLM-KG extracts hierarchical features from KG representations and performs inter-layer and intra-layer interactions in each iteration. The features from interactions enter the joint reasoning module, where each QA context feature and KG feature mutually attend to each other. The representations of the two modalities are fused and updated through multi-step interactions. Finally, using the information provided by the interaction layer, irrelevant nodes in the KG are removed. Experiments conducted on the commonsense datasets CommonsenseQA, OpenbookQA, and the medical question and answer dataset MedQA-USMLE show that the performance on the MedQA-USMLE dataset is superior to baseline models, and on other datasets, the performance is close to baseline models, demonstrating its competitiveness in terms of reasoning ability.