Scholarlink

<p>Graph neural networks (GNNs) face critical challenges when processing non-Euclidean data, including node feature over-smoothing, inadequate handling of heterogeneous graphs, and difficulties in modeling long-range dependencies. Graph transformers (GTs) offer novel solutions by synergistically integrating local graph structures with global attention mechanisms; however, their practical implementation in large-scale graph applications remains constrained by quadratic computational complexity. In this study, FuseFormer, which is a compact GTs architecture specifically optimized for node classification tasks that efficiently fuses graph-structural information and attention mechanisms through innovative design, is proposed. The core innovation lies in the linear differential transform (LDT) implementation using the neighborhood-constrained Frobenius norm as the kernel function. FuseFormer maintains linear computational complexity while enabling cost-effective deep integration of graph-structural features with transformer mechanisms, thereby simultaneously addressing over-globalization issues through differential computation principles. Furthermore, the introduced StarNet architecture facilitates shallow-wide network design that restricts the information propagation range, thereby effectively mitigating node feature over-smoothing. Experimental results demonstrate the superiority of FuseFormer over GTs methods across homogeneous, heterogeneous, and large-scale graphs, with accuracy improvements of 4.71–5.84%. Notably, our framework achieved a classification accuracy of 66.46% on the ogbn-papers100M benchmark.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

FuseFormer: deep fusion of GNNs and linear differential transformers for large-scale graphs

  • 期刊论文

  • 发表日期：

    2026年5月21日

查看全文
  • Zhu Liu,
  • Peng Wang,
  • Cui Ni,
  • Guangyuan Zhang,
  • Xiaojie Hu,
  • Zhongzheng Zhen

摘要

Graph neural networks (GNNs) face critical challenges when processing non-Euclidean data, including node feature over-smoothing, inadequate handling of heterogeneous graphs, and difficulties in modeling long-range dependencies. Graph transformers (GTs) offer novel solutions by synergistically integrating local graph structures with global attention mechanisms; however, their practical implementation in large-scale graph applications remains constrained by quadratic computational complexity. In this study, FuseFormer, which is a compact GTs architecture specifically optimized for node classification tasks that efficiently fuses graph-structural information and attention mechanisms through innovative design, is proposed. The core innovation lies in the linear differential transform (LDT) implementation using the neighborhood-constrained Frobenius norm as the kernel function. FuseFormer maintains linear computational complexity while enabling cost-effective deep integration of graph-structural features with transformer mechanisms, thereby simultaneously addressing over-globalization issues through differential computation principles. Furthermore, the introduced StarNet architecture facilitates shallow-wide network design that restricts the information propagation range, thereby effectively mitigating node feature over-smoothing. Experimental results demonstrate the superiority of FuseFormer over GTs methods across homogeneous, heterogeneous, and large-scale graphs, with accuracy improvements of 4.71–5.84%. Notably, our framework achieved a classification accuracy of 66.46% on the ogbn-papers100M benchmark.