Hand gesture recognition has become increasingly essential, not only for facilitating communication for hearing-impaired individuals but also for implementing automation that minimizes human contact with surfaces, especially in the post-pandemic era. However, the implementation of accurate recognition systems faces several challenges due to variations in human hand size, varying distances from the camera, and hand orientations, making it difficult to establish a direct correspondence between reference gestures and target gestures. To address these challenges, this research employs a machine learning (ML)-based technique that emulates the human brain’s ability to identify gestures in a user-independent manner, regardless of distance, palm size, and orientation. In this research work, the pre-trained MediaPipe hand model has been used to extract hand landmark points from the hand data corpus. These landmark points, along with hand-face position, were utilized to develop feature vectors for the classification process. After normalization, these features exhibited a high correlation within the same category of hand gestures. Initially, a spot-checking activity has been done to recognize the machine learning model that can accurately identify gestures from the normalized feature vectors. The spot-checking process uses eight different machine learning models and they have been fed with a subset of total available hand gestures from the corpus. It has been identified that the multi-layer perceptron (MLP) ensures 100% classification accuracy that motivates the development of the final model using the MLP classification algorithm with 25 different hand gestures. This developed model was able to ensure a 99.62% accuracy level when it was tested with data collected from more numbers of hand profiles.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Person-Independent Hand Gesture Recognition Using MediaPipe and Multi-layer Perceptron

  • Subhajyoti Barman,
  • Sejuti Majumdar

摘要

Hand gesture recognition has become increasingly essential, not only for facilitating communication for hearing-impaired individuals but also for implementing automation that minimizes human contact with surfaces, especially in the post-pandemic era. However, the implementation of accurate recognition systems faces several challenges due to variations in human hand size, varying distances from the camera, and hand orientations, making it difficult to establish a direct correspondence between reference gestures and target gestures. To address these challenges, this research employs a machine learning (ML)-based technique that emulates the human brain’s ability to identify gestures in a user-independent manner, regardless of distance, palm size, and orientation. In this research work, the pre-trained MediaPipe hand model has been used to extract hand landmark points from the hand data corpus. These landmark points, along with hand-face position, were utilized to develop feature vectors for the classification process. After normalization, these features exhibited a high correlation within the same category of hand gestures. Initially, a spot-checking activity has been done to recognize the machine learning model that can accurately identify gestures from the normalized feature vectors. The spot-checking process uses eight different machine learning models and they have been fed with a subset of total available hand gestures from the corpus. It has been identified that the multi-layer perceptron (MLP) ensures 100% classification accuracy that motivates the development of the final model using the MLP classification algorithm with 25 different hand gestures. This developed model was able to ensure a 99.62% accuracy level when it was tested with data collected from more numbers of hand profiles.