Adaptive Deep Graph Convolutional Network for Dialogical Speech Emotion Recognition

With the increasing demand for humanization of human-computer interaction, dialogical speech emotion recognition (SER) has attracted the attention from researchers, and it is more aligned with actual scenarios. In this paper, we propose a dialogical SER approach that includes two modules, first module is a pre-trained model with an adapter, the other module is the proposed adaptive deep graph convolutional network (ADGCN). Since emotional speech data, especially dialogical data is scarce, it is difficult for models to extract enough information through limited data. A self-supervised pre-trained framework Data2vec is introduced. An adapter is designed to integrate pre-trained model to reduce the training cost while maintaining its extensive speech-related knowledge. The representations learned by adapter are from independent utterances (intra-utterance level), and ADGCN is proposed to model dialogical contextual information in one dialogue (inter-utterance level). Two residual mechanisms, adaptive residual and dynamic local residual are designed in the proposed ADGCN, which keep it from over-smoothing issues when increasing the number of layers to model global inter-utterance level contextual information. All experiments in this paper are conducted on IEMOCAP and achieved 76.49% in term of weighted accuracy.