Scene Sino-Nom Text Optical Character Recognition in Vietnam: A Novel Approach Combining Deep Learning and Linguistic Knowledge
摘要
Sino-Nom script (a combination of Chinese and Nom scripts) holds a significant place in Vietnamese cultural history, capturing the thoughts, events, values, and traditions of the nation in documents such as books, edicts, and decrees, as well as in external inscriptions like couplet, horizontal lacquered boards, and relics. These texts are widely found at historical and cultural sites, including temples, pagodas, shrines, and mausoleums throughout Vietnam. However, most modern Vietnamese cannot read Sino-Nom script. Thus, converting Sino-Nom text from outdoor photographs into readable text using optical character recognition (OCR) has become an urgent and meaningful task. This method plays a vital role in preserving, restoring, and propagating Vietnamese cultural heritage to the present generation through smart applications. This paper pioneers the study of outdoor Sino-Nom character recognition in Vietnam, a highly challenging task due to the intricate script structures, diverse environmental conditions, and limited data resources. Leveraging recent advancements in deep learning and integrating linguistic knowledge, we propose the first outdoor Sino-Nom character recognition system in Vietnam. Our contributions include an OCR method applied to the first annotated dataset of Sence Sino-Nom text, collected from historical and cultural relics, temples, pagodas, shrines, and mausoleums across Vietnam. The proposed system combines deep learning models with phonetic linguistic knowledge to enhance recognition accuracy. Experimental results show that our approach achieves an 80% accuracy rate, demonstrating its effectiveness. This work establishes a foundation for future research on the digital preservation of Sino-Nom characters and practical applications in cultural heritage technology.