Graph isomorphism attention network combined with pre-trained language models: a novel approach for crystal material property prediction
摘要
Predicting crystal material properties is a central task in AI-for-science, yet existing graph neural network methods face three limitations: (1) conventional neighbor-based graph construction causes node features to be dominated by neighbors, losing atomic uniqueness, (2) such atomic graphs only capture short-range dependencies via immediate neighbors while neglecting long-range interactions through multi-hop connections, and (3) insufficient utilization of chemical formulae, often relying on simplistic one-hot encoding without material-specific knowledge. To address these, we propose the graph isomorphism attention network (GIAT), which balances self-node and neighbor information to preserve atomic characteristics while capturing short-range dependencies. Additionally, we integrate a GraphTransformer to model long-range atomic interactions, forming a complementary framework with GIAT. Furthermore, we employ matBERT, a material science-specific language model, to encode chemical formulae, leveraging domain knowledge from both individual materials and their analogs. Experiments show that our model achieves state-of-the-art performance by synergistically combining short-range (GIAT), long-range (GraphTransformer), and chemical formula embeddings (matBERT).