A Systematic Review of Multimodal Emotion Recognition Approaches for Affective Computing: Advances and Challenges
摘要
Affective computing as an exciting interdisciplinary domain is delving into psychology, neuroscience, and computer science aims to enhance the field of human-computer interaction. It is enabling machines to act more naturally in recognizing and responding to human emotions. This paper marks the significant foray into multimodal emotion recognition approach in affective computing. It underscores the inherent constraints of unimodal systems while accentuating the escalating significance and applicability of multimodal datasets. This systematic review delves into advancement of multimodal emotion recognition approach, by analysing 120 papers from prominent databases such as Scopus, IEEE, Elsevier and Google Scholar. This comprehensive review remarks the advancement in multimodal emotion recognition approach, including data fusion methods, offering profound insights into their practical challenges. In conclusion, this review highlights critical concerns such as dataset diversity, cultural biases, and the complexity of integrating heterogeneous data from various modalities. A systematic review of existing literature underscores the continuous evolution of deep learning in refining data level fusion strategies as well as feature level fusion strategies. This will enhance the integration of multimodal data for emotion recognition in emotionally aware devices. However, a significant limitation lies in the challenge of effectively aligning heterogeneous data sources. This will continue to hinder progress and underscores the need for more advanced adaptive fusion techniques and robust modeling approaches.