Improving Efficiency and Performance Through CTC-Based Transformers for Mathematical Expression Recognition

Mathematical expression recognition (MER) is an active field with important implications for educational technology, document analysis, and scientific research automation. The bi-dimensionality and complexity of mathematical expressions presents challenges in accurately interpreting expressions, such as distinguishing subscripts from superscripts and comprehending structures like fractions and matrices, which require robust recognition systems that integrate spatial understanding and structural awareness. In this paper, we address the problem of typeset mathematical expression recognition using a convolutional neural network backbone model and a transformer encoder trained with connectionist temporal classification loss. Compared to state-of-the-art systems, this encoder-only model proves highly efficient, achieving a speed-up factor of 3.5 and 4.2 for the IBEM and Im2Latex-100k datasets. Moreover, this approach outperforms most state-of-the-art systems on mark-up-level and image-level metrics across both datasets. A comprehensive study demonstrates the model’s capability to interpret complex reading orders of mathematical expressions, showing that the monotonicity of the CTC alignments is not a limitation of CTC-based models for the problem of MER. These findings underscore the effectiveness of this approach when compared to auto-regressive methods.