Speech impairment, characterised by the inability to speak and hear, necessitates the use of alternate communication methods, such as sign language to facilitate effective interaction. Sign language in Malaysia is not widely due to the small deaf community and lack of awareness among the general population. Providing assistance to the deaf in public interactions requires training and specialized skills. Therefore, the primary objective of this work is to create a reliable system for recognizing static gesture of Malaysian Sign Language (MSL). This project employs a pre-trained deep neural network model, namely YOLO (You Only Look Once) architecture to analyse MSL movements. The methodology involves real-time image capturing, labelling, and preprocessing to ensure precise gesture localization and classification. The model was trained over 100 epochs, optimizing performance by minimizing losses related to localization, confidence, and classification. This method aims to enhance communication accessibility for those with speech impairments. The proposed model demonstrated its maximum precision, recall and F1 score, nearing 1.0 with high confidence level for all classes of MSL signs used in the experiment. This confirms the model's capability to achieved an optimal balance between precision and recall in the classification process, laying strong foundation for advancement in MSL recognition.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Malaysian Sign Language Recognition for Static Gestures Using Transfer Learning Technique

  • Siti Nisrina Imtinan Tanjung,
  • Nik Adilah Hanin Zahri,
  • Nur Hafizah Ghazali,
  • Amiza Amir,
  • Rusnida Romli,
  • Nik Mohd Zarifie Hashim

摘要

Speech impairment, characterised by the inability to speak and hear, necessitates the use of alternate communication methods, such as sign language to facilitate effective interaction. Sign language in Malaysia is not widely due to the small deaf community and lack of awareness among the general population. Providing assistance to the deaf in public interactions requires training and specialized skills. Therefore, the primary objective of this work is to create a reliable system for recognizing static gesture of Malaysian Sign Language (MSL). This project employs a pre-trained deep neural network model, namely YOLO (You Only Look Once) architecture to analyse MSL movements. The methodology involves real-time image capturing, labelling, and preprocessing to ensure precise gesture localization and classification. The model was trained over 100 epochs, optimizing performance by minimizing losses related to localization, confidence, and classification. This method aims to enhance communication accessibility for those with speech impairments. The proposed model demonstrated its maximum precision, recall and F1 score, nearing 1.0 with high confidence level for all classes of MSL signs used in the experiment. This confirms the model's capability to achieved an optimal balance between precision and recall in the classification process, laying strong foundation for advancement in MSL recognition.