Learning structure perception MLPs on graphs: a layer-wise graph knowledge distillation framework

Graph neural networks (GNNs) are expressive in dealing with graph data. Because of the large storage requirements and the high computational complexity, it is difficult to deploy these cumbersome models in resource-constrained environments. As a representative model compression strategy, knowledge distillation (KD) is introduced into graph analysis research to address this problem. However, there are some crucial challenges in existing graph knowledge distillation algorithms, such as knowledge transfer effectiveness and student model designation. To address these problems, a new graph distillation model is proposed in this paper. Specifically, a layer-wise mapping strategy is designed to distill knowledge for training the student model, in which staged knowledge learned by intermediate layers of teacher GNNs is captured to form supervision signals. And, an adaptive weight mechanism is developed to evaluate the importance of the distilled knowledge. On this basis, a structure perception MLPs is constructed as the student model, which can capture prior information of the input graph from the perspectives of node feature and topology structure. In this way, the proposed model shares the prediction advantage of GNNs and the latency advantage of MLPs. Node classification experiments on five benchmark datasets demonstrate the validity and superiority of our model over baseline algorithms.