Multi-hop Segmentation for Knowledge Graph Reasoning Based on Deep Reinforcement Learning

Due to their ability to provide interpretable decisions, walk-based models have gained significant advantages in automatically completing various reasoning tasks over incomplete knowledge graphs. However, achieving good results on long path queries with multi-hop reasoning based on single-agent is difficult, and the reasoning performance degrades significantly with increasing path length. The key challenges of multi-hop reasoning on long paths are graph sparsity and the incompleteness of the information collected by the agent. In this paper, we propose a simple yet efficient multi-hop segmentation framework for knowledge graph reasoning that mitigates performance fluctuations. Our method is modeled as an offline reinforcement learning Markov decision model. It converts multi-hop reasoning to a combination of multiple single-hop reasoning, wherein the path information closer to the target entity is hidden at the current timestep. We use the multi-head attention mechanism in Transformer to process the current action space. To address entities with larger fan-in and fan-out and the problem of false paths in the later stage of long-path reasoning, we propose an equal probability action dropout policy that discards the current actions. Experiments on three large datasets (FB15K-237, WN18RR, and NELL-995) show that our method outperforms all single-agent and dual-agent models significantly.