Multi-language: ensemble learning-based speech emotion recognition

Inaccurate emotional reactions from robots have been a problem for authors in previous years. Since technology has advanced, robots like service robots can communicate with people of many other languages. The traditional Speech Emotion Recognition (SER) method utilizes the same corpus for classifier testing and training to accurately identify emotions. However, this method could be more flexible for multi-lingual (multi-language) contexts, which is essential for robots that people use worldwide. This research proposes an ensemble learning method (HMLSTM and CapsNet) that uses a voting majority for a cross-corpus, multi-lingual SER system. This work utilizes three corpora (EMO-DB, URDU, and SAVEE) that offer a variety of languages (German, Urdu, and English) to test multi-language SER. We first use the Refined Attention Pyramid Network (RAPNet) for speech and emotion recognition to extract the features. Following that, the pre-processing step of the data is normalized using the Min–max normalization approach and IGAN to address data imbalance. To identify the emotions in speech into the appropriate group, use HMLSTM and CapsNet’s ensemble learning algorithms. With reasonable accuracy, the proposed ensemble learning approach enhances emotion recognition. It compares the effectiveness of the proposed ensemble learning method with existing traditional learning methods. Using data from a corpus trained on a different corpus, this study tests the performance of a classifier for multi-lingual emotion identification. In this experiment, distinct classifiers offer excellent accuracy for diverse corpora.