Speech emotion recognition with Wav2vec and deep gated recurrent units
摘要
Automatic emotion recognition has a wide range of applications, including call centre monitoring, automotive experience analysis, and patient emotion state analysis, all of which fall under affective computing. A significant challenge in this field is the scarcity of annotated data, particularly for real-time, multilingual emotion recognition. Traditional emotion recognition models, such as Dialogue RNN and Conversational Memory Networks, rely heavily on textual features, limiting their applicability when only acoustic information is available. In this work, the authors proposed a novel approach that utilizes only acoustic features to achieve state-of-the-art results. It reduces the need for text by using transfer learning from Wav2Vec embeddings and Gated Recurrent Units (GRUs) as our main model. Our method is evaluated on the RAVDESS dataset, achieving an overall accuracy of 79.82%. Furthermore, the model demonstrates high recall for emotions such as Neutral (67%), Calm (61%), Happy (60%), Sad (66%), Angry (75%), Fearful (63%), Disgust (60%), and Surprised (68%). Compared to Bi-LSTM and CNN-Bi-LSTM, our proposed model achieves superior performance with a precision of 0.65, a recall of 0.64, an accuracy of 79.8% and an F1 score of 0.64. To further assess the proposed DNN-3 model, the author performed 10-fold cross-validation, added noise to the RAVDESS dataset and executed DNN-3 on the TESS and JL Corpus datasets.