CRL-TIKG: Causal Reinforcement Learning-Based Threat Intelligence Knowledge Graph Reasoning
摘要
Threat Intelligence Knowledge Graph (TIKG) construction and reasoning are constrained by fragmented, heterogeneous Cyber Threat Intelligence (CTI) and poor semantic integration, insufficient for the analysis of complex attack scenarios, such as Advanced Persistent Threats (APTs). Existing TIKG reasoning, reliant on statistical correlations, suffers from semantic conflicts, inadequate causal modeling, and poor interpretability. To address these limitations and shortcomings, we propose CRL-TIKG, a novel framework integrating Causal Reinforcement Learning (CRL) for advanced TIKG reasoning, completion, and causal discovery. First, we employ Large Language Models (LLMs), guided by the MITRE ATT&CK framework, to construct a semantically rich, tactic-level TIKG from diverse intelligence sources, enhancing semantic consistency. Second, we develop a tactic-level Causal Discovery Model (CDM) based on Topological Hawkes Processes (THP), anchored to ATT&CK tactics, to delineate causal dependencies between attack stages. Finally, we train a CRL agent using Proximal Policy Optimization (PPO), which leverages the TIKG and CDM for multi-hop reasoning under a novel causal-aware reward function. CRL-TIKG mitigates multi-source semantic conflicts through semantically enhanced graph construction, precisely captures attack causal relationships via the CDM, and enhances reasoning interpretability with a causal reward mechanism. Experimental results show that CRL-TIKG has a precision of 84% on the TIKG entity-relations recognition task, and the percentage of strong causal edges reaches 93%, which outperforms existing methods.