Spectral Topology Meets Local Geometry: A Dual-Block Graph Transformer with Provable Expressive Power
摘要
Traditional Transformers struggle with graph tasks due to their inability to jointly model spectral topology (global structural invariants) and local geometric structure. While existing approaches either restrict attention to local neighborhoods-sacrificing the Transformer’s capacity to learn global patterns-or overemphasize global attention at the expense of graph-aware locality, we bridge this divide through a theoretically grounded framework. We propose the Dual-Block Graph Transformer (DBGT), a novel architecture that synergizes spectral topology and local geometry via two co-designed blocks: (1) a Message-Passing Transformer (MPT) block enforcing edge-aware local attention, and (2) a global attention block operating to preserve global invariants. Crucially, DBGT achieves this integration without computational overhead by unifying the two blocks into a single-head design with provable expressive power. Our theoretical analysis shows that this dual-block approach can capture complex graph signals with high expressive power. Experiments on 10 benchmark datasets, including heterophilic graphs, validate DBGT’s superiority: it achieves state-of-the-art accuracy of 94.12% (Photo) and 89.19% (Texas), outperforming both GNNs and Graph Transformers. Notably, DBGT attains these results without complex positional encodings, underscoring its inherent capability to model graph structure.