Building a speech emotion recognition system using RNN, GRU and LSTM
摘要
The Speech Emotion Recognition (SER) task is a challenging problem in the area of affective computing, and it needs strong approaches to identify emotional states from acoustic signals. In this work, we examine the performance of recurrent neural networks (RNNs) composed of simple RNNs, Gated Recurrent Units (GRUs) and Long Short-Term Memory networks (LSTMs) with 13 coefficients of Mel-Frequency Cepstral Coefficients (MFCCs). We evaluate these architectures on three benchmark data sets: Toronto Emotional Speech Set (TESS), Surrey Audio-Visual Expressed Emotion (SAVEE), and Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). The experiment setup consists of four steps: single data set testing, pair-wise fusion (RAVDESS + TESS, SAVEE + TESS, SAVEE + RAVDESS), and full three-data set fusion. Results indicate that LSTM networks excel on homogeneous datasets, achieving 99.62% accuracy on TESS, while GRUs excel with regards to robustness on more challenging, smaller datasets like SAVEE. Results highlight the central role of the relationship between dataset characteristics (size, speaker variability, and emotional distinction) and model selection for optimal SER performance.