Constructing Personal Knowledge Graph from Conversation via Deep Reinforcement Learning

Due to the rise of intelligent assistants, a massive amount of user QA(question-and-answer) data has emerged on the Internet. This data contains valuable information about user concerns and preferences in specific domains. However, there has been limited research exploring the user-related information contained in such conversational data for constructing individual user knowledge graphs. We propose a method for learning to construct a personal knowledge graph from multi-turn QA data between agents and users. We build a reinforcement learning-based knowledge graph path reasoning model. This model maps user utterances to an action space composed of paths between entity nodes, enabling path reasoning on the graph and facilitating traversal to the next entity node. The approach utilizes emotional and intent information inferred from subsequent user responses as a reward signal for training the reinforcement learning policy network. We conduct experiments on the ConvRef dataset, consisting of 11k naturally occurring dialogues, and compare our method with state-of-the-art baselines. The results demonstrate that our approach effectively generates more accurate inference paths from user-agent dialogue interactions and constructs high-quality personal knowledge graphs.