Multimodal Sentiment Analysis Using Deep Learning: A Review

Multimodal Sentiment Analysis (MSA) is a burgeoning field in natural language processing (NLP), also known as opinion mining. It determines sentiment(positive, negative, neutral), subjective opinion, emotional tone, sometimes even more fine-grained emotion like joy, anger, sadness, and others. The evolution of sentiment analysis from its early days of text only analysis to the incorporation of multimodal data has significantly enhanced the accuracy and depth of sentiment understanding. MSA is poised to play a pivotal role in extracting valuable insights from the vast amount of multimodal data generated in today’s digital age. Various fusion methods have been developed to combine information from different modalities effectively. Additionally, the field has seen significant contributions from lexical-based, machine learning-based, and deep learning-based approaches. Deep learning, in particular, has revolutionized MSA by enabling the creation of complex models that can effectively analyze sentiment from diverse data sources. This survey provides an overview of the critical developments in MSA, highlighting the evolution of methods. It also presents a comparative analysis of state-of-the-art models and their performance on benchmark datasets and future potential, helping researchers and practitioners choose the most suitable approach for their specific tasks. The surveyed models SKEAFN, TEDT, UniMSE, MMML and others have exhibited impressive performance across various datasets.