Lip reading holds a significant importance in enhanced communication, supplement of sign language, improved social interaction, emergency situations, etc. This topic focuses on detecting speech from video frames without audio. Earlier studies show that automatic lip-reading systems have been developed for languages like Chinese, Korean, English, and German. Research indicates that developing a system for lip-reading from video frames without audio input is challenging due to factors such as lighting, shooting distance, and gender. Lip-reading systems were initially created using traditional machine learning techniques. Recent research suggests that deep learning-based lip-reading produces better results than traditional methods. In our work we used advanced deep learning techniques to interpret spoken words from lip movements. Through the integration of convolutional neural networks (CNNs) and bidirectional long short-term memory (LSTM) networks, the system ensures user-friendly accessibility for individuals with limited technical expertise. The 3D CNN layers intricately capture lip movements, while the LSTM layers enhance the system’s contextual understanding. Training is centered on establishing connections between lip movements and words, with a clear emphasis on simplicity to facilitate user comprehension of the complexities of deep learning. Precision is included in around 95%. As a result, the 3D-CNN for the GRID dataset outperforms the preceding system in terms of classification accuracy. This groundbreaking approach transforms speech comprehension by leveraging visual cues, leading to the development of more intuitive communication systems.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Visual Speech Recognition: Advancements in Lip-Reading Through Deep Learning

  • Kandula Narasimharao,
  • Uppalapati Padma Jyothi,
  • Chinta Gowthami,
  • J. Harini

摘要

Lip reading holds a significant importance in enhanced communication, supplement of sign language, improved social interaction, emergency situations, etc. This topic focuses on detecting speech from video frames without audio. Earlier studies show that automatic lip-reading systems have been developed for languages like Chinese, Korean, English, and German. Research indicates that developing a system for lip-reading from video frames without audio input is challenging due to factors such as lighting, shooting distance, and gender. Lip-reading systems were initially created using traditional machine learning techniques. Recent research suggests that deep learning-based lip-reading produces better results than traditional methods. In our work we used advanced deep learning techniques to interpret spoken words from lip movements. Through the integration of convolutional neural networks (CNNs) and bidirectional long short-term memory (LSTM) networks, the system ensures user-friendly accessibility for individuals with limited technical expertise. The 3D CNN layers intricately capture lip movements, while the LSTM layers enhance the system’s contextual understanding. Training is centered on establishing connections between lip movements and words, with a clear emphasis on simplicity to facilitate user comprehension of the complexities of deep learning. Precision is included in around 95%. As a result, the 3D-CNN for the GRID dataset outperforms the preceding system in terms of classification accuracy. This groundbreaking approach transforms speech comprehension by leveraging visual cues, leading to the development of more intuitive communication systems.