Scholarlink

<p>Speech Emotion Recognition (SER) remains a challenging task in human-computer interaction due to its reliance solely on vocal cues, which are often affected by variability in expression, background noise, and imbalanced datasets. This study proposes a deep learning-based SER system enhanced by data augmentation techniques, including noise injection, spectrogram shifting, and speed perturbation. The system was evaluated on three benchmark datasets: RAVDESS, EMODB, and TESS. Class imbalance in RAVDESS and EMODB was addressed using the Synthetic Minority Oversampling Technique (SMOTE). The model leverages both prosodic and spectral speech features. Two deep learning architectures were tested: a Deep Neural Network (DNN) and a Transformer Encoder. The Transformer-based model achieved superior performance, reaching 99.30% accuracy on EMODB, 92.00% on RAVDESS, and 100% on TESS–yielding improvements of +1.2%, +1.8%, and +0.04% respectively over the DNN baseline. Real-world testing on unseen recordings further demonstrated the Transformer’s superior generalization capabilities. To validate its practical deployment, a Streamlit-based application was developed. These findings highlight the potential of Transformer-based architectures–when combined with data augmentation and feature integration–to deliver accurate and reliable emotion recognition in real-world speech scenarios.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Transformer encoder and data augmentation for real-time speech emotion recognition

  • 期刊论文

  • 发表日期：

    2026年2月26日

查看全文
  • Chawki Barhoumi,
  • Yassine BenAyed

摘要

Speech Emotion Recognition (SER) remains a challenging task in human-computer interaction due to its reliance solely on vocal cues, which are often affected by variability in expression, background noise, and imbalanced datasets. This study proposes a deep learning-based SER system enhanced by data augmentation techniques, including noise injection, spectrogram shifting, and speed perturbation. The system was evaluated on three benchmark datasets: RAVDESS, EMODB, and TESS. Class imbalance in RAVDESS and EMODB was addressed using the Synthetic Minority Oversampling Technique (SMOTE). The model leverages both prosodic and spectral speech features. Two deep learning architectures were tested: a Deep Neural Network (DNN) and a Transformer Encoder. The Transformer-based model achieved superior performance, reaching 99.30% accuracy on EMODB, 92.00% on RAVDESS, and 100% on TESS–yielding improvements of +1.2%, +1.8%, and +0.04% respectively over the DNN baseline. Real-world testing on unseen recordings further demonstrated the Transformer’s superior generalization capabilities. To validate its practical deployment, a Streamlit-based application was developed. These findings highlight the potential of Transformer-based architectures–when combined with data augmentation and feature integration–to deliver accurate and reliable emotion recognition in real-world speech scenarios.