Information Extraction and Knowledge Acquisition (IEKA) aims to distill knowledge from unstructured data, and hypergraph neural networks (HGNNs), with their advantage in high-order relation modeling, can effectively handle complex semantic relationships. However, the high-order modeling capability of hypergraphs also brings a significant increase in computational complexity, which becomes the main obstacle in practical industrial deployment. When seeking a balance between computational efficiency and expressive power, Knowledge Distillation (KD) is an ideal solution. Existing research typically uses MultiLayer Perceptrons (MLPs) as student models, but MLPs perform poorly in terms of interpretability and accuracy. We propose a novel knowledge distillation framework: HGNN2KAN, where HGNN2KAN adopts the Kolmogorov-Arnold Network (KAN) as the student model, and HGNN2KAN+ further extracts high-reliability node information to supervise the training of the student model. Experimental results on eight hypergraph datasets show that HGNN2KAN+ improves accuracy by 1.11% compared to LightHGNN+ using MLP as the student model, and by 10.44% compared to the student model KAN. Compared to HGNNs, its inference time is reduced by 44 times. This study demonstrates that HGNN2KAN can achieve performance comparable to HGNNs while significantly reducing computational complexity, making it suitable for scenarios with strict low-latency requirements.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

HGNN2KAN: Distilling Hypergraph Neural Networks Into KAN for Efficient Inference

  • Junzheng Li,
  • Hongtao Yu,
  • Ruiyang Huang,
  • Yawei Ning,
  • Suchang Yang

摘要

Information Extraction and Knowledge Acquisition (IEKA) aims to distill knowledge from unstructured data, and hypergraph neural networks (HGNNs), with their advantage in high-order relation modeling, can effectively handle complex semantic relationships. However, the high-order modeling capability of hypergraphs also brings a significant increase in computational complexity, which becomes the main obstacle in practical industrial deployment. When seeking a balance between computational efficiency and expressive power, Knowledge Distillation (KD) is an ideal solution. Existing research typically uses MultiLayer Perceptrons (MLPs) as student models, but MLPs perform poorly in terms of interpretability and accuracy. We propose a novel knowledge distillation framework: HGNN2KAN, where HGNN2KAN adopts the Kolmogorov-Arnold Network (KAN) as the student model, and HGNN2KAN+ further extracts high-reliability node information to supervise the training of the student model. Experimental results on eight hypergraph datasets show that HGNN2KAN+ improves accuracy by 1.11% compared to LightHGNN+ using MLP as the student model, and by 10.44% compared to the student model KAN. Compared to HGNNs, its inference time is reduced by 44 times. This study demonstrates that HGNN2KAN can achieve performance comparable to HGNNs while significantly reducing computational complexity, making it suitable for scenarios with strict low-latency requirements.