Adaptive feature fusion for scene text script identification

Script identification is an essential preliminary step in multilingual OCR systems. This paper focuses primarily on tackling the challenging problem of script identification in scene text images, which are usually characterized by low image quality, diverse text styles, and complex backgrounds. Furthermore, script identification becomes a fine-grained classification problem when some scripts share common characters. To address this issue, we propose a novel end-to-end CNN comprising two streams for extracting distinct types of features, namely, visual features and spatial features. In the visual stream, we introduce an enhanced Squeeze-and-Excitation (SE) channel attention mechanism to emphasize valuable features and suppress irrelevant ones. The enhanced SE is composed of squeeze and excitation steps. The squeeze step employs adaptive average pooling for information aggregation. Two 1x1 convolutional layers are used to derive channel weights in the excitation step. In the spatial stream, we perform efficient analysis of the spatial dependencies within the text lines based on LSTM. Finally, we propose an adaptive fusion approach that combines probability vectors from the two streams. Instead of being fixed, the weight assigned to each probability vector is learned during network training. To validate our proposed method, we conduct extensive tests on four publicly available datasets, viz. MLe2e, RRC-MLT2017, SIW-13, and CVSI-2015. Our proposed method achieves accuracies of \(97.66\%\) $97.66\%$ , \(90.24\%\) $90.24\%$ , \(96.66\%\) $96.66\%$ , and \(98.44\%\) $98.44\%$ on these four datasets, respectively, which compare favorably with state-of-the-art methods. The two streams have demonstrated complementarity. Moreover, ablation experiments have been conducted to verify the effectiveness of each component in the proposed method.