Speech emotion recognition using the novel SwinEmoNet (Shifted Window Transformer Emotion Network)

Understanding human emotions is necessary for various tasks, including interpersonal interaction, knowledge acquisition, and determining courses of action. Recognizing emotions, particularly in speech, poses significant challenges due to linguistic differences, local differences, diversity in gender identities, generational differences, and diversity of cultures. Deep learning methods are promising for automating this task; previous approaches frequently rely on a single type of feature representation, limiting the efficacy of Speech Emotion Recognition (SER). To address these limitations, a comprehensive approach that uses Shifted Window Transformers is proposed, which considers the many different aspects of emotional expression in speech and the use of diverse feature representations to improve SER performance. This paper outlines a novel, Shifted Windowed Transformer Emotional Network (SwinEmoNet), incorporating shifted window attention mechanisms for efficient emotion classification. SwinEmoNet uses local window attention rather than traditional transformer architectures’ global attention mechanisms. This capability allows the model to concentrate essential data in small, finer sections of the input speech signal. The proposed SwinEmoNet architecture has been evaluated against three distinct speech spectrograms. This paper deals with the effectiveness of the proposed SER method by analyzing its performance on the Berlin Emotional Database (EMODB) and the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS), with an emphasis on metrics such as accuracy, precision, recall and F1-score. With the EMODB and RAVDESS datasets, the accuracy of the SwinEmoNet is 94.93 and 96.51%, respectively, significantly outperforming existing transformer models and current state-of-the-art standards.