Scholarlink

<p>Human–Computer Interaction (HCI) depends critically on hand gesture recognition, which also provides the basis of sign language communication for those with hearing problems. In automatic sign language identification systems, however, the contribution of non-manual elements like head posture and facial emotions has sometimes been disregarded. In order to fully capture the diversity of sign language modalities and to enhance the quality of life of deaf people in the region of Ha’il, we present in this study a dual-stream convolutional neural network (TS-CNN) that separately models hand motions and head positions. We provide fusion techniques that improve feature correlation and representation in order to guarantee efficient cooperation between the two streams. Moreover, we use a Feature Enhancement Module (FEM) to enhance gloss-level prediction accuracy and fine temporal alignment. Two benchmark datasets, RWTH-PHOENIX-Weather 2014 and CSL Split II, are used to assess the TS-CNN framework and show a notable increase in recognition performance. Competitive Word Error Rates (WER) obtained by TS-CNN from experimental data validate its performance in simulating the complicated multimodal character of continuous sign language recognition.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A dual-stream deep learning framework for continuous sign language recognition to enhance communication accessibility in the Ha’il region

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月3日

查看全文
  • Haifa Harrouch,
  • Hanene Guesmi,
  • Hany Alalfy,
  • Maher Jebali

摘要

Human–Computer Interaction (HCI) depends critically on hand gesture recognition, which also provides the basis of sign language communication for those with hearing problems. In automatic sign language identification systems, however, the contribution of non-manual elements like head posture and facial emotions has sometimes been disregarded. In order to fully capture the diversity of sign language modalities and to enhance the quality of life of deaf people in the region of Ha’il, we present in this study a dual-stream convolutional neural network (TS-CNN) that separately models hand motions and head positions. We provide fusion techniques that improve feature correlation and representation in order to guarantee efficient cooperation between the two streams. Moreover, we use a Feature Enhancement Module (FEM) to enhance gloss-level prediction accuracy and fine temporal alignment. Two benchmark datasets, RWTH-PHOENIX-Weather 2014 and CSL Split II, are used to assess the TS-CNN framework and show a notable increase in recognition performance. Competitive Word Error Rates (WER) obtained by TS-CNN from experimental data validate its performance in simulating the complicated multimodal character of continuous sign language recognition.