To address the long-standing lack of data resources and modeling challenges in Uyghur lip-reading, this study constructs the first Uyghur word-level lip-reading dataset (ULR), comprising 4,000 video samples from 20 speakers. We design two dataset partitioning schemes to evaluate model generalization ability under known and unknown speaker conditions. This dataset fills the resource gap in visual speech recognition for low-resource languages and provides a solid foundation for subsequent algorithmic research and cross-language transfer learning. Targeting the linguistic characteristics of Uyghur—characterized by subtle lip movements and challenging phoneme discrimination—we introduce spatial attention based on ECA-Net and design an efficient channel-spatial attention module (ECA-S) to enhance the model’s ability to capture key visual regions and feature channels. Experimental results demonstrate that the ECA-S model achieves competitive accuracy of 89.08% on the public LRW dataset and exhibits superior generalization performance under both partitioning schemes of the ULR dataset. Our research not only provides the first standardized data resources for Uyghur lip-reading but also opens new avenues for low-resource language lip-reading research and applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Towards Uyghur Lip-Reading: Dataset Development and Attention-Enhanced Recognition with ECA-S Module

  • Siyuan Wei,
  • Zilong Xing,
  • Mutallip Mamut,
  • Kurban Ubul

摘要

To address the long-standing lack of data resources and modeling challenges in Uyghur lip-reading, this study constructs the first Uyghur word-level lip-reading dataset (ULR), comprising 4,000 video samples from 20 speakers. We design two dataset partitioning schemes to evaluate model generalization ability under known and unknown speaker conditions. This dataset fills the resource gap in visual speech recognition for low-resource languages and provides a solid foundation for subsequent algorithmic research and cross-language transfer learning. Targeting the linguistic characteristics of Uyghur—characterized by subtle lip movements and challenging phoneme discrimination—we introduce spatial attention based on ECA-Net and design an efficient channel-spatial attention module (ECA-S) to enhance the model’s ability to capture key visual regions and feature channels. Experimental results demonstrate that the ECA-S model achieves competitive accuracy of 89.08% on the public LRW dataset and exhibits superior generalization performance under both partitioning schemes of the ULR dataset. Our research not only provides the first standardized data resources for Uyghur lip-reading but also opens new avenues for low-resource language lip-reading research and applications.