According to the World Health Organization (2024) [1], 430 million people (including 34 million children) have a deafness problem. This means that more than 5% of the world's population needs rehabilitation services. Sign Language Recognition (SLR) is an important technology that helps communication between deaf and hearing people. Current SLR systems have difficulties with accuracy, scalability, and they need a large computing capacity. In this paper we will introduce a new approach for SLR using keypoints instead of RGB videos and a hybrid RNN model for robust sequence prediction. With this new approach, we will need less computing capacity, making it accessible on a wider range of devices. Our solution is implemented in a mobile, a desktop and a web application. The dataset used in this study is collected by our team in collaboration with Al Fath Association for Deaf Children of Meknes [2]. This dataset includes over than 28 class of Moroccan Sign Language (MSL). Each class contain multiple videos for the same word. Those videos are recorded by multiple individuals under varying conditions to capture a wider variety of signing styles and environments. Extensive testing demonstrated the effectiveness of our approach, characterized by high accuracy and low computational costs. This solution has the potential to enhance communication for Moroccan deaf individuals and contribute to the advancement of SLR technologies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An Innovative Solution for Predicting Moroccan Sign Language Using Mediapipe Keypoints and Deep Learning

  • Touzouz Adnane,
  • Bekri Ali,
  • Farah Rachid

摘要

According to the World Health Organization (2024) [1], 430 million people (including 34 million children) have a deafness problem. This means that more than 5% of the world's population needs rehabilitation services. Sign Language Recognition (SLR) is an important technology that helps communication between deaf and hearing people. Current SLR systems have difficulties with accuracy, scalability, and they need a large computing capacity. In this paper we will introduce a new approach for SLR using keypoints instead of RGB videos and a hybrid RNN model for robust sequence prediction. With this new approach, we will need less computing capacity, making it accessible on a wider range of devices. Our solution is implemented in a mobile, a desktop and a web application. The dataset used in this study is collected by our team in collaboration with Al Fath Association for Deaf Children of Meknes [2]. This dataset includes over than 28 class of Moroccan Sign Language (MSL). Each class contain multiple videos for the same word. Those videos are recorded by multiple individuals under varying conditions to capture a wider variety of signing styles and environments. Extensive testing demonstrated the effectiveness of our approach, characterized by high accuracy and low computational costs. This solution has the potential to enhance communication for Moroccan deaf individuals and contribute to the advancement of SLR technologies.