ISLNet: A Landmark-Driven Framework for Accurate and Scalable Indian Sign Language Recognition
摘要
This research presents a real-time ISL recognition system designed to aid the deaf and mute people in communicating with non-signers. It uses the Kaggle ISL dataset, consisting of 35 static gesture categories and 42,000 images. Using MediaPipe, hand landmarks are extracted and converted into 84-dimensional relative normalized vectors centered at the wrist, ensuring robustness to hand size, position, and orientation. One of the key innovations is the use of landmark-based features along with a lightweight neural network that can be optimized for real-time performance on low-resource devices. Conventional image-based models rely on raw pixels and large convolutional networks, but this system uses vectorized representations for faster and more efficient computation without compromising accuracy. A Feedforward Neural Network with Dropout and L2 regularization is trained on the above vectors to improve generalization and prevent overfitting. The model achieves 98.62% test accuracy, with 99.01% precision, 98.62% recall, and a 98.69% F1-score. In real-time prediction, OpenCV captures frames and MediaPipe detects hand landmarks, leading to responsive gesture classification. With a frame rate of 5–6 FPS, the system is well-suited for static gesture recognition using mobiles or other embedded platforms. During live predictions, confidence scores are also shown, which offers immediate user feedback, making the system practical and scalable for real-world use.