MetaSelection: A Learnable Masked AutoEncoder for Multimodal Sentiment Feature Selection

Multimodal learning has demonstrated a great advantage in sentimental analysis tasks due to the richer information from different modalities, especially the complementary information. However, our study shows that multimodal data not only provides useful complementary information, but also contains some information that is irrelevant to or conflicts with the task of sentiment prediction. It can degrade the training effectiveness of multimodal models. To tackle this problem, we propose a Learnable Masked AutoEncoder (LMAE) to eliminate the irrelevant or conflicting features of each modality by a learned mask. Afterward, the selected features from modalities are fused by a cross-modal attention. Experiments on samples with conflicting information across modalities and two benchmark datasets, CMU-MOSI and CMU-MOSEI, demonstrate the superiority of our proposal over seven state-of-the-art methods.