Real Time Arabic Sign Language Recognition Using Machine Learning: A Vision - Based Approach

Dumb and hearing-impaired people are unable to communicate as well as normal people; thus, they must rely on sign language. Which is a visual or gestural form of communication. Unfortunately, sign language is neither common nor easy to learn. Consequently, deaf-mute people encounter many challenges in their daily communication. Hence, we proposed an intelligent system that employs a vision-based approach dedicated to Arabic sign language recognition (ArSLR). The system is aimed to recognize Arabic words expressed in dynamic sign language expressions and translate them into textural form. While maintaining natural and flexible translation, the system doesn’t impose any hardware requirements, colored gloves, or limitations on the background. A custom dataset is utilized in the development process of the system. A significant amount of experimental work has been conducted to come up with effective and generalized algorithms for image processing, feature extraction, feature selection, and practical classification. Satisfactory results are achieved with the use of linear discriminant analysis (LDA) as a feature selection and dimensionality reduction method; where 100% accuracy is achieved with support vector machine (SVM) and logistic regression classification algorithms. An accuracy of 99.9% is obtained with a multilayer perceptron (MLP) feed-forward artificial neural network and 99.6% with a decision tree classification. The impact of using principal component analysis (PCA) and LDA as a feature selection approach instead of raw data features is also presented. Finally, the system is validated in real-time with unseen data. It is evident from the validation and testing results that word-level ArSLR can be achieved robustly and accurately.