A Hybrid Approach Using Convolution and Transformer for Mongolian Ancient Documents Recognition

Mongolian ancient documents are an indispensable source for studying Mongolian traditional culture. To thoroughly explore and effectively utilize these ancient documents, conducting a comprehensive study on Mongolian ancient document words recognition is essential. In order to better recognize the word images in Mongolian ancient documents, this paper proposes an approach that combines convolutional neural networks with Transformer models. The approach used in this paper takes word images as the input for the model. After passing through a feature extractor composed of convolutional neural networks, the extracted features are fed into a Transformer model for prediction. Finally, the corresponding recognition results of the word images are obtained. Due to the common existence of imbalanced distribution of character classes in recognition tasks, models often tend to excessively focus on common characters while neglecting rare characters. Our proposed approach integrates focal loss to enhance the model’s attention towards rare characters, thereby improving the overall recognition performance of the model for all characters. After training, the model is capable of rapidly and efficiently performing end-to-end recognition of words in Mongolian ancient documents. The experimental results indicate that our proposed approach outperforms existing methods for word recognition in Mongolian ancient documents, effectively improving the performance of Mongolian ancient document words recognition.