TransTg: a new transformer model for predicting glass transition temperature of polymers from monomers’ molecular structures

The escalating demand for novel and multifunctional polymers with superior properties requires a deeper exploration of the chemical space, coupled with a well-established understanding of the relationship between their chemical structure and properties. The glass transition temperature (Tg), representing the temperature at which a polymer transitions from a solid to a soft and pliable state, is one important characteristic of polymers. Thus, predicting Tg values is crucial for discovering novel polymeric materials for diverse applications. However, experimental determination can be costly and time-consuming besides relying on specific experimental conditions that severely affect the measured Tg values. Recently, data-driven models based on machine learning have emerged as promising tools for predicting Tg values using molecular structure information. In this work, a novel SMILES-to-Tg transformer (TransTg) is developed as a new transformer-based deep learning model to predict the Tg values of polymers. Transformer models exploit the self-attention mechanism that captures long-range dependencies in sequential representations, which is suitable for modeling complex polymer structures. Our proposed transformer model achieves notable performance, with a mean absolute error (MAE) of 22.55 and a coefficient of determination (R²) of 0.849 on a held-out test set, outperforming the state-of-the-art results on using QSPR and machine learning approaches on monomers’ molecular structure. Besides their impressive performance, transformer-based models offer the advantage of result interpretation through their self-attention mechanism. Unlike black-box machine learning models, this mechanism highlights the most critical parts of the input sequence during the learning process, thereby aiding in result interpretation. By relating chemical language (SMILES), and chemical properties, there exists potential for developing language-to-property transformer models that directly link monomer structures to various polymer properties across multiple domains and tasks.