Data-free Knowledge Distillation based on GNN for Node Classification
摘要
Data-free Knowledge Distillation (KD) circumvents the limitation of knowledge extraction from original training data by utilizing generated data. Data-free KD has made good progress in models for processing grid data. However, for Graph Neural Networks (GNN) that process non-grid data, existing related methods are primarily designed for graph classification tasks involving small-sized graphs. Research on node classification tasks for larger graphs remains unexplored. To address this issue, we propose the first Data-Free Knowledge Distillation framework for Node Classification (DFKD-NC). DFKD-NC obtains training data by generating target node and its neighbor information. For data generation, we predefine a full tree pseudo-subgraph template for each target node and use a generator component to generate node features. We train student model and generator using adversarial training. Moreover, we introduce contrastive learning to diversify the generated data and regularize the generated node features to facilitate student model convergence. Extensive experiments on six benchmark datasets have demonstrated that DFKD-NC achieves state-of-the-art results in node classification tasks. Code is available at https://github.com/zengxinfeng/DFKD-NC.