Accent identification from emotional speech using classification fusion of multiple deep learning models
摘要
Accent identification systems have been developed for read or neutral speech. It is essential to build an accent identification system from emotional speech since human speech embeds appropriated emotions to convey the intended content and certain differences in speaker accents. Detecting accents from emotional speech is very important for developing robust speech emotion recognition system (SER) and automatic speech recognition system (ASR). In this paper, we present a novel analysis based on the classification fusion of multiple deep learning models for accent identification systems from emotional speech. The presented method utilizes deep learning models, such as convolutional neural networks (CNN), bidirectional long short-term memory (BLSTM), and deep neural networks (DNN). The CNN is built using spectrogram images of speech utterances, the BLSTM is trained on short-term low-level features, and the long-term statistical features are estimated using DNN. Finally, a decision score-based classification fusion strategy is applied to integrate multiple classifier results for identifying accents from emotional speech. The Crema-D emotional speech dataset is used for evaluation as it has accented speech samples. The performance of the presented system is reported and compared with the individual use of such deep learning models and multiple machine learning classifiers trained using spectral features. Experimental results show that jointly modelling with multiple deep learning models improves accent identification performance on emotional speech.