Scholarlink

<p>This study delves into the intricacies of automatic lip reading (ALR) in the Turkish language, employing a deep learning model that integrates convolutional neural networks (CNNs) and long short-term memory (LSTM) units. By analyzing a comprehensive dataset comprising 111 words and 113 sentences, encompassing 67,080 video samples, the research identifies several key challenges inherent in Turkish ALR. The introduction of a novel "Sentences with Derived Words" (SDW) dataset underscores the impact of Turkish agglutinative morphology on ALR performance. Through rigorous analysis of commonly misclassified words, word lengths, phonetic resemblances, and consonant–vowel interactions, the study reveals that morphological diversity, phonetic similarities, and word lengths significantly influence model accuracy. Words containing bilabial consonants exhibit higher recognition rates, whereas shorter words and those with similar structures are more prone to misclassification. The SDW dataset, in particular, highlights the challenges posed by derived words, as evidenced by a notable decrease in word recognition rate. This work not only identifies critical limitations but also proposes recommendations for improving ALR models in Turkish, emphasizing the incorporation of phonetic features and the enhancement of dataset diversity. The findings offer profound insights into advancing ALR technologies for diverse applications, including assistive communication, forensic analysis, and human–computer interaction.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Challenges and enhancements in Turkish automatic lip reading using deep learning models

  • 期刊论文

  • 发表日期：

    2026年3月31日

查看全文
  • Furkan Sabaz,
  • Ümit Atila,
  • Murat Dörterler,
  • Alaettin Uçan

摘要

This study delves into the intricacies of automatic lip reading (ALR) in the Turkish language, employing a deep learning model that integrates convolutional neural networks (CNNs) and long short-term memory (LSTM) units. By analyzing a comprehensive dataset comprising 111 words and 113 sentences, encompassing 67,080 video samples, the research identifies several key challenges inherent in Turkish ALR. The introduction of a novel "Sentences with Derived Words" (SDW) dataset underscores the impact of Turkish agglutinative morphology on ALR performance. Through rigorous analysis of commonly misclassified words, word lengths, phonetic resemblances, and consonant–vowel interactions, the study reveals that morphological diversity, phonetic similarities, and word lengths significantly influence model accuracy. Words containing bilabial consonants exhibit higher recognition rates, whereas shorter words and those with similar structures are more prone to misclassification. The SDW dataset, in particular, highlights the challenges posed by derived words, as evidenced by a notable decrease in word recognition rate. This work not only identifies critical limitations but also proposes recommendations for improving ALR models in Turkish, emphasizing the incorporation of phonetic features and the enhancement of dataset diversity. The findings offer profound insights into advancing ALR technologies for diverse applications, including assistive communication, forensic analysis, and human–computer interaction.