A periodicity aware transformer for crystal property prediction

Crystals constitute a variety of important materials from everyday life to cutting-edge fields. The properties of a crystal are determined by its structure, as demonstrated by physics theory, which is essential for understanding and designing materials. In recent years, deep learning-based methods have been proposed to predict crystal material properties and achieved satisfactory performance. However, these methods have not adequately accounted for the key composition of crystals, i.e., periodicity. To address this issue, we propose a periodicity aware crystal transformer (PACT), which uses hierarchical self-attention mechanisms to enforce periodicity constraints on the crystal structure. Specifically, it applies unit-wise self-attention and crystal-wise self-attention to ensure that the surroundings of atoms or unit cells at periodic distances are identical. Extensive benchmark experiments demonstrate that our proposed model exhibits superior performance, achieving an average improvement of 7.07% over previous methods. Additionally, ablation studies show both unit-wise self-attention and crystal-wise in the hierarchical self-attention mechanisms are effective in modeling the periodicity.