Scholarlink

<p>Recognizing emotions is crucial for developing human–computer interaction (HCI) systems and related digital products. Due to the rising demand for such applications, deep learning models for recognizing emotions in speech have become a significant area of research. The objective of this work is to develop a robust multilingual SER framework capable of accurately recognizing emotions from speech signals across different languages. However, most speech emotion recognition systems have been developed for other languages, leaving low-resource languages like Kannada underrepresented. To address this gap, the Kannada Speech Emotion (KES) database has been utilized, and various Data Augmentation (DA) methods have been used to expand the database, which will enhance the performance, generalization, and robustness of the Speech Emotion Recognition (SER) model. Motivated by advancements in Deep learning, this work focuses on a Multi-Lingual Speech emotion recognition system for Kannada, English, and German using KES, EMO-DB, and RAVDESS datasets, respectively. The paper proposes a hybrid CNN-LSTM-GRU model with an attention mechanism with a feature-level fusion vector of three spectral parameters (chromagram, MFCC and log mel-scaled spectrogram), and three prosody parameters (ZCR, RMS value). The experimental results demonstrate the efficacy and robustness of the proposed SER model when compared with the SOTA SER methods and achieved up to 99.4%, 98.2%, and 96.8% accuracy over augmented RAVDESS, EMO-DB, and KES databases, respectively. The results obtained from augmented and original databases have been compared to highlight the significance of data augmentation in SER. This work highlights the effectiveness of a hybrid model for SER and provides a groundwork for future research in low-resource languages.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multilingual Speech Emotion Recognition using Hybrid Convolution Neural Network with Attention Mechanism

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月1日

查看全文
  • Geethashree A.,
  • Ramesh B. T.,
  • Chandrashekar M. Patil,
  • Audre Arlene Anthony

摘要

Recognizing emotions is crucial for developing human–computer interaction (HCI) systems and related digital products. Due to the rising demand for such applications, deep learning models for recognizing emotions in speech have become a significant area of research. The objective of this work is to develop a robust multilingual SER framework capable of accurately recognizing emotions from speech signals across different languages. However, most speech emotion recognition systems have been developed for other languages, leaving low-resource languages like Kannada underrepresented. To address this gap, the Kannada Speech Emotion (KES) database has been utilized, and various Data Augmentation (DA) methods have been used to expand the database, which will enhance the performance, generalization, and robustness of the Speech Emotion Recognition (SER) model. Motivated by advancements in Deep learning, this work focuses on a Multi-Lingual Speech emotion recognition system for Kannada, English, and German using KES, EMO-DB, and RAVDESS datasets, respectively. The paper proposes a hybrid CNN-LSTM-GRU model with an attention mechanism with a feature-level fusion vector of three spectral parameters (chromagram, MFCC and log mel-scaled spectrogram), and three prosody parameters (ZCR, RMS value). The experimental results demonstrate the efficacy and robustness of the proposed SER model when compared with the SOTA SER methods and achieved up to 99.4%, 98.2%, and 96.8% accuracy over augmented RAVDESS, EMO-DB, and KES databases, respectively. The results obtained from augmented and original databases have been compared to highlight the significance of data augmentation in SER. This work highlights the effectiveness of a hybrid model for SER and provides a groundwork for future research in low-resource languages.