This research presents a real-time ISL recognition system designed to aid the deaf and mute people in communicating with non-signers. It uses the Kaggle ISL dataset, consisting of 35 static gesture categories and 42,000 images. Using MediaPipe, hand landmarks are extracted and converted into 84-dimensional relative normalized vectors centered at the wrist, ensuring robustness to hand size, position, and orientation. One of the key innovations is the use of landmark-based features along with a lightweight neural network that can be optimized for real-time performance on low-resource devices. Conventional image-based models rely on raw pixels and large convolutional networks, but this system uses vectorized representations for faster and more efficient computation without compromising accuracy. A Feedforward Neural Network with Dropout and L2 regularization is trained on the above vectors to improve generalization and prevent overfitting. The model achieves 98.62% test accuracy, with 99.01% precision, 98.62% recall, and a 98.69% F1-score. In real-time prediction, OpenCV captures frames and MediaPipe detects hand landmarks, leading to responsive gesture classification. With a frame rate of 5–6 FPS, the system is well-suited for static gesture recognition using mobiles or other embedded platforms. During live predictions, confidence scores are also shown, which offers immediate user feedback, making the system practical and scalable for real-world use.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ISLNet: A Landmark-Driven Framework for Accurate and Scalable Indian Sign Language Recognition

  • Anupam Agrawal,
  • Ishan Goel,
  • Ashutosh Rajpal

摘要

This research presents a real-time ISL recognition system designed to aid the deaf and mute people in communicating with non-signers. It uses the Kaggle ISL dataset, consisting of 35 static gesture categories and 42,000 images. Using MediaPipe, hand landmarks are extracted and converted into 84-dimensional relative normalized vectors centered at the wrist, ensuring robustness to hand size, position, and orientation. One of the key innovations is the use of landmark-based features along with a lightweight neural network that can be optimized for real-time performance on low-resource devices. Conventional image-based models rely on raw pixels and large convolutional networks, but this system uses vectorized representations for faster and more efficient computation without compromising accuracy. A Feedforward Neural Network with Dropout and L2 regularization is trained on the above vectors to improve generalization and prevent overfitting. The model achieves 98.62% test accuracy, with 99.01% precision, 98.62% recall, and a 98.69% F1-score. In real-time prediction, OpenCV captures frames and MediaPipe detects hand landmarks, leading to responsive gesture classification. With a frame rate of 5–6 FPS, the system is well-suited for static gesture recognition using mobiles or other embedded platforms. During live predictions, confidence scores are also shown, which offers immediate user feedback, making the system practical and scalable for real-world use.