A Novel Approach for Recognition and Classification of Hand Gesture Using Deep Convolution Neural Networks

Sign language is a means of communication between individuals, whether they are hard of hearing or other people who do not speak the language of the host country. While it is a means of communication, few people know it and it has no universal patterns. For example, there is a set of signs unique to each country, resulting from the customs and traditions of each country. Currently, much research is being done in this area to solve the problems of sign language translation using computer vision and artificial intelligence. The importance of this topic is due to the possibility of using Deep Convolutional Neural Networks (CNNs), embedded in deep learning technology, to recognize hand gestures in real time and to translate sign language. This article presents two new datasets containing 54,049 of images of the ArSL-2018 (Arabic Sign Language) alphabet taken by more than 40 people, as well as 15,200 images in our personal datasets, categorized into 32 classes of standard Arabic characters, and classified, normalized, and detected using a VGGNet model and ResNet50. In this study, the success of two different training and testing exercises performed without fine-tuning and with fine-tuning enhancers was compared. The optimized weights of each VGGNet layer were achieved as the network was pre-trained on two large datasets of alphabet sign language. In addition, the ResNet50 classifier was trained using 40 epochs and fine-tuned with 40 plus epochs to ensure optimal classification performance. The high accuracy levels achieved in comparison with other studies support the effectiveness of this approach. Specifically, the ArSL alphabets dataset achieved accuracies of 99,05%, 99,99%, and 98,50%, using VGG16, VGG19, and ResNet50 Models respectively, demonstrating the effectiveness of the proposed method for hand gesture recognition tasks.