Heterogeneous Graph Neural Network Based on Self-knowledge Distillation and Transformer

Heterogeneous Graph Neural Networks (HGNNs) have garnered significant attention in recent years. They are particularly effective at modeling complex structures and diverse node types in real-world data. However, effectively capturing both structural and semantic information in heterogeneous graphs while maintaining computational efficiency remains a critical challenge. In this work, we propose HTKD, a Heterogeneous Transformer with self-Knowledge Distillation framework. The model employs a graph structural encoder that captures topological relationships and node interactions, and a graph semantic encoder that extracts rich, context-aware semantics. These complementary embeddings are then adaptively fused via a Transformer to generate expressive node-level representations. To further enhance representation learning while reducing computational overhead, we incorporate a self-knowledge distillation mechanism, allowing deeper semantic insights to be distilled into shallower layers. Extensive experiments on multiple benchmark datasets demonstrate that our approach achieves superior performance. The code is publicly available at https://github.com/404LEO404/HTKD .