Scholarlink

<p>Gesture-based biometric systems are emerging as a promising approach for secure, natural, and intuitive human–computer interaction in pervasive environments. However, unimodal gesture-based systems are inherently limited by modality-specific vulnerabilities, such as sensitivity to noise, occlusions, and reduced robustness under variability in execution, which can compromise security and reliability. While surface electromyography (EMG) and 3D skeletal motion capture have individually been explored, their systematic multimodal fusion remains under-investigated, despite its potential to enhance robustness and biometric security. In this work, we present a novel multimodal dataset that synchronously records 8-channel EMG signals from forearm muscles together with 3D hand skeleton data from a Leap Motion Controller. The dataset comprises multiple participants performing three distinct gestures (wave, fist, thumbs-up) enabling systematic evaluation across authentication and recognition tasks. Experimental results demonstrate that for person authentication, unimodal classifiers based on EMG and skeleton data achieve accuracies of 95.6% and 92.5%, respectively, while classifier-level fusion boosts performance to 99.4%, underscoring the complementary nature of the two modalities. Authentication performance was further validated under a verification (1:1) protocol using standard biometric measures (EER and ROC-AUC), confirming strong separability between genuine and impostor attempts beyond identification accuracy alone. For gesture recognition, the skeleton modality alone attains near-perfect accuracy (99.16%), whereas EMG alone is comparatively modest (68.20%, macro-F1 0.68). Feature-level fusion yields a small but consistent gain over Skeleton (99.61% accuracy), while classifier-level fusion underperforms Skeleton (94.06%), indicating that naïve late fusion can dilute a dominant modality without calibration or reliability weighting. These findings establish a task-dependent best practice: late fusion for authentication; Skeleton-centric recognition with optional feature-level fusion for robustness. Beyond introducing a benchmark dataset, this study provides empirical insights into designing secure and user-friendly multimodal interaction systems, with implications for pervasive applications in AR/VR, rehabilitation, and next-generation biometric security.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Multimodal benchmark of EMG and 3D hand motion for gesture recognition and biometric authentication

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月27日

查看全文
  • Wejdan Al-Mari,
  • Hamda Al-Marri,
  • Sheyma Al-Jaber,
  • Alreem Al-Tamimi,
  • Jayakanth Kunhoth,
  • Somaya Al-Maadeed,
  • Moutaz Saleh,
  • Younes Akbari

摘要

Gesture-based biometric systems are emerging as a promising approach for secure, natural, and intuitive human–computer interaction in pervasive environments. However, unimodal gesture-based systems are inherently limited by modality-specific vulnerabilities, such as sensitivity to noise, occlusions, and reduced robustness under variability in execution, which can compromise security and reliability. While surface electromyography (EMG) and 3D skeletal motion capture have individually been explored, their systematic multimodal fusion remains under-investigated, despite its potential to enhance robustness and biometric security. In this work, we present a novel multimodal dataset that synchronously records 8-channel EMG signals from forearm muscles together with 3D hand skeleton data from a Leap Motion Controller. The dataset comprises multiple participants performing three distinct gestures (wave, fist, thumbs-up) enabling systematic evaluation across authentication and recognition tasks. Experimental results demonstrate that for person authentication, unimodal classifiers based on EMG and skeleton data achieve accuracies of 95.6% and 92.5%, respectively, while classifier-level fusion boosts performance to 99.4%, underscoring the complementary nature of the two modalities. Authentication performance was further validated under a verification (1:1) protocol using standard biometric measures (EER and ROC-AUC), confirming strong separability between genuine and impostor attempts beyond identification accuracy alone. For gesture recognition, the skeleton modality alone attains near-perfect accuracy (99.16%), whereas EMG alone is comparatively modest (68.20%, macro-F1 0.68). Feature-level fusion yields a small but consistent gain over Skeleton (99.61% accuracy), while classifier-level fusion underperforms Skeleton (94.06%), indicating that naïve late fusion can dilute a dominant modality without calibration or reliability weighting. These findings establish a task-dependent best practice: late fusion for authentication; Skeleton-centric recognition with optional feature-level fusion for robustness. Beyond introducing a benchmark dataset, this study provides empirical insights into designing secure and user-friendly multimodal interaction systems, with implications for pervasive applications in AR/VR, rehabilitation, and next-generation biometric security.