Speech Emotion Recognition Based on Multi Acoustic Feature Fusion
摘要
Speech emotion recognition (SER) is an essential part of human-computer interaction. Speech contains rich acoustic information, and it is difficult to accurately predict real emotions based on a single acoustic feature. This paper proposes a new SER model that utilizes multiple acoustic features and designs a novel fusion mechanism. Multiple speech features are first extracted: spectrograms, MFCC, and raw speech waveforms, which are encoded separately using AlexNet encoder, LSTM encoder, and the emotion2vec model. Then, a fusion mechanism proposed in this paper is used to fuse these features to obtain the final emotion prediction. Experiments conducted on the IEMOCAP dataset demonstrate that our proposed model achieves competitive results, with 72.07% WA and 72.65% UA in the leave-one-session-out validation strategy, and 74.93% WA and 75.80% UA in the leave-one-speaker-out validation strategy.