Multilingual Speech Emotion Recognition Using Deep Learning Approach

Human emotion is an inherent part of human beings, and it is used to express their feelings to the listeners. While emotions are mostly conveyed via facial expressions, spoken words also contain emotions to reflect a speaker’s emotional state. This project focused on researching and evaluating the deep neural network performance on multi-lingual speech emotion recognition on RAVDESS, EMO-DB and combination of both emotional speech databases. Methodology used in the project was divided into five steps: data collection and speech signal extraction, signal conversion, image recognition using transfer learning, result validation and implementation of trained network in graphical user interface (GUI). The research on AlexNet and SqueezeNet in transfer learning was carried out by training the networks using different number of maximum epochs, learning rate and image augmentations. The research showed that AlexNet provided the higher validation accuracy than SqueezeNet at 66.20% during training the combined RAVDESS and EMO-DB databases. As for the testing data, the trained model obtained an F1-score of 0.6253 on testing 264 sample data.