Multi-head multi-order graph attention networks

The Graph Attention Network (GAT) is a type of graph neural network (GNN) that uses attention mechanisms to weigh the importance of nodes’ neighbors, demonstrating flexibility and power in representation learning. However, GAT and its variants still face common challenges in GNNs, such as over-smoothing and over-squashing. To address this, we propose Multi-Head Multi-Order Graph Attention Networks (MHMOGAT) as an enhanced GAT layer. MHMOGAT is built based on multi-head attention and adjacency matrices of different orders, aiming to expand the receptive field of GAT to effectively capture long-distance dependencies. Moreover, Bayesian optimization is employed to determine optimal hyperparameter combinations for different datasets. Experimental results on six prevailing datasets demonstrate that MHMOGAT improves GAT accuracy by approximately 2-5% across various datasets with different label rates, indicating its effectiveness. Additionally, MHMOGAT exhibits potential in handling large and complex graphs with low label rates.