The Impact of Data Augmentation Techniques on Improving Generalization Ability and Accuracy of Speech Emotion Recognition

Emotions play a vital role in human–computer interface (HCI) applications. Accurately identifying and interpreting emotions from a speaker’s voice is crucial for various real-time applications that aim to enhance user experience and interaction. One of the main challenges in SER is the availability of balanced and diverse training data. Imbalanced datasets can lead to biased models and poor generalization of unseen data. To address this issue, the paper explores the effectiveness of data augmentation (DA) techniques. Two approaches have been proposed, the first approach utilizes convolutional neural networks (CNNs), which are widely used for analyzing sequential data such as speech signals. However, the paper acknowledges that CNNs may struggle to learn optimal features from complex speech signals alone. To address this limitation, the second approach was suggested to employ convolutional long-term memory (ConvLSTM). ConvLSTM combines the power of both CNNs and long short-term memory (LSTM) networks, allowing the model to capture both spatial and temporal dependencies in the speech signals. The study findings provide empirical evidence supporting the efficacy of the proposed approach, especially the ConvLSTM-based approach, in achieving high accuracy for speech emotion recognition. Remarkable accuracies of 94.44%, 97.7%, and 97.19% are achieved for the RAVDESS, SAVEE, and EMO-DB datasets, respectively.