Music Emotion Classification Algorithm Based on Multimodal Deep Learning

In response to the problem of existing algorithms that only consider single-modal features of music or videos, leading to low classification efficiency, this paper proposes a music emotion classification algorithm based on multimodal deep learning. Firstly, a two-dimensional audio convolutional neural network is used, and Mel spectrograms are utilized to learn these audio features. Then, we employ multimodal fusion techniques to integrate audio and video features and design a multimodal deep learning classification algorithm for music emotion classification. This paper constructs a diverse music video dataset to address the lack of labelled music video datasets. Experimental validation is conducted on these datasets to verify the effectiveness of the proposed algorithm, and the impact of different optimizers on the performance of single-modal classification models is compared and analyzed. The experimental results demonstrate that the multimodal classifier achieves the best classification performance compared to the single-modal emotion classifier.