An Effective CNN Based Indian Sign Language Recognition System Using Federated Learning

Sign language is a visual form of communication used by deaf and hearing-impaired people. It relies on hand gestures, body language, and facial emotions. Recognition of dynamic sign language is a more challenging problem compared to recognition of static sign language as it often involves sequence of multiple frames. This necessitates a significant memory resource to handle video dataset and high processing power for training the model. This paper presents a dynamic Indian Sign Language (ISL) recognizer system to recognize fifteen dynamic signs. It is built based on the federated learning approach, which utilizes three clients with identically distributed data and a single server system connected through a network for communication. The System takes hand gestures from video frames as input and process them using a custom trained 2-D Convolution Neural Network (2D CNN) model which predicts dynamic sign gestures and translates them into a respective natural language (audio). Experimental results demonstrate that the use of the Federated Learning approach reduces the memory and network requirements in distributed model training, resulting in an 87% accuracy.