Dictionary Temporal Graph Network via Pre-training Embedding Distillation

In temporal graph learning that focuses on the evolution of graphs over time, dictionary temporal graph network is an emerging and important technology. Compared with existing methods, the dictionary network can be stored, updated, and calculated more flexibly during training, thus achieving better performance. However, it still has some problems: (1) Such dictionary network relies heavily on reliable original features, lacking these, it will bring cold starting problems; and (2) during the constant updating process of embeddings, the problem of catastrophic forgetting can arise. To solve these problems, this paper proposes Dictionary Temporal graph network via pre-training Embedding Distillation (DicTED). Specifically, by introducing pre-trained teacher models to generate prior embeddings, the above problems can be solved: (1) The combination of prior embeddings with original features can ensure the model obtains reliable node embeddings on the input side; and (2) the design of contrastive loss can solve the catastrophic forgetting problem on the optimization side, which ensures that the model can find a balance between learning new knowledge and old knowledge. Experiments on multiple real-world datasets demonstrate that DicTED achieves better performance than existing SOTA methods. The code and data are available at: https://github.com/Samer-hue/DicTED .