Boosting Pre-trained Language Models for Temporal Knowledge Graph Reasoning via Joint Structure and Recurring Patterns

Temporal Knowledge Graph (TKG) reasoning aims to infer missing facts based on historical information. Pre-trained Language Models (PLMs) have excelled in knowledge-intensive tasks, motivating numerous studies to explore their semantic modeling capability for TKG reasoning. However, existing PLM-based methods primarily focus on temporal semantic modeling while neglecting structural dependencies and global historical patterns, which limits their effectiveness. To address these challenges, we propose PSG, a PLM-based framework that takes into account Structure and Global information for TKG reasoning. Specifically, we design specific contextualized time-aware soft prompts to combine and balance semantic and structural information, which are then processed by both the frozen PLMs and the graph model. Besides, based on our analysis, we capture timely and relevant global information by utilizing global frequency, which emphasizes interactions involving recent historical facts. We evaluate PSG on three datasets and conduct extensive analysis, validating the effectiveness of PSG in TKG reasoning.