MIGCL: Fake news detection with multimodal interaction and graph contrastive learning networks

Due to the rapid growth of news containing multimedia elements, such as images in social networks, cross-modal learning is crucial for accurate fake news detection. Most previous approaches focus on embedding images and sentences independently into a shared embedding space by developing complex neural networks to coarsely fuse multimodal information. However, these approaches rarely seek fine-grained connections between images and sentences prior to performing multimodal fusion and lack the ability to understand complex intra- and intermodal relationships. In addition, previous studies have primarily concentrated on intra- and intermodal relationships within each sample, but interclass sample dynamics have been neglected. To address these issues, we propose a multimodal interaction and graph contrastive learning network (MIGCL) for fake news detection. The multimodal interaction network consists of cross-modal alignment and filtering mechanisms that take into account both locally fine-grained and comprehensive cross-modal interactions while also adaptively suppressing irrelevant cross-modal interactions. Moreover, we develop a hierarchical graph contrastive learning framework that employs fully and self-supervised contrastive learning methods to investigate the intricate connections of intra- and intermodal representations. More precisely, unimodal graphs are constructed at the intramodal level to explore the authenticity information contained in the intra- and interclass samples of a particular modality. At the intermodal level, multimodal graphs are constructed to capture the correlations between intra- and interclass cross-modal samples. Furthermore, we enhance the robustness of the model feature representation by applying perturbations to the graph structure. The proposed MIGCL achieves superior performance on three benchmark datasets, indicating the efficacy of our approach.