Speech emotion recognition (SER) plays a vital role in enhancing human-computer interaction by enabling machines to understand and respond to human emotional states. However, traditional air conducted (AC) speech-based methods often face. Bone conducted (BC) speech, which captures vibrations transmitted through the skull, offers a promising. Despite its potential, existing research on BC speech-based SER lacks comprehensive evaluation using extensive performance metrics and thorough analysis across emotion classes and speaker demographics. This study proposes a deep learning model integrating convolutional neural networks and transformer-based attention mechanisms to effectively capture both local and global acoustic patterns from BC speech signals. The model is trained and evaluated on a benchmark BC speech dataset with rigorous data preprocessing and feature extraction techniques including MFCC and Mel spectrograms. Extensive hyperparameter tuning and validation confirm the model’s capacity for generalization. Key findings demonstrate that the proposed approach achieves a validation accuracy of 96.99% and training accuracy of 97.03%, supported by strong metrics such as a balanced accuracy of 96.98%, macro F1-Score of 0.97 and Matthews correlation coefficient of 0.9656. These results confirm the effectiveness and robustness of BC speech for emotion recognition, addressing existing research gaps and paving the way for practical emotion-aware applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Emotion Recognition from Bone-Conducted Speech Using Attention-Based CNN-Transformer Architecture on the EmoBone Dataset

  • Md. Ibne Shihab Shad,
  • Sayeeda Khan,
  • Md. Sarwar Hosain,
  • Akif Mahdi,
  • Rinku Islam,
  • Liton Chandra Paul

摘要

Speech emotion recognition (SER) plays a vital role in enhancing human-computer interaction by enabling machines to understand and respond to human emotional states. However, traditional air conducted (AC) speech-based methods often face. Bone conducted (BC) speech, which captures vibrations transmitted through the skull, offers a promising. Despite its potential, existing research on BC speech-based SER lacks comprehensive evaluation using extensive performance metrics and thorough analysis across emotion classes and speaker demographics. This study proposes a deep learning model integrating convolutional neural networks and transformer-based attention mechanisms to effectively capture both local and global acoustic patterns from BC speech signals. The model is trained and evaluated on a benchmark BC speech dataset with rigorous data preprocessing and feature extraction techniques including MFCC and Mel spectrograms. Extensive hyperparameter tuning and validation confirm the model’s capacity for generalization. Key findings demonstrate that the proposed approach achieves a validation accuracy of 96.99% and training accuracy of 97.03%, supported by strong metrics such as a balanced accuracy of 96.98%, macro F1-Score of 0.97 and Matthews correlation coefficient of 0.9656. These results confirm the effectiveness and robustness of BC speech for emotion recognition, addressing existing research gaps and paving the way for practical emotion-aware applications.