Fake News Detection by Incorporating Multi-modal Information

Multi-modal expressions in social media contain richer information and have a wider spreading effect of fake news. Therefore, how to effectively use multi-modal information to accurately extract the representation features of fake news and detect them timely has become an urgent research task to be solved. Compared with single-text content, the difficulties faced by multi-modal fake news detection tasks mainly include: (1) extracting pertinent features to accurately represent fake news across various modalities poses a challenge; (2) there is a lack of unified feature representation methods to correlate multi-modal features such as text and image. To address these challenges, we propose a Multi-modal Pretrained Model (MPM) for detecting fake news by incorporating multimodal information. Extensive compared experiments were conducted on a multimedia dataset from Twitter named MediaEval2015. The experimental results demonstrate that the detection accuracy of MPM reached 91.8%, which is 21.8% better than the unimodal detection method and 41.7% better than the multimodal baseline model. The results verified the feasibility of incorporating multimodal information and the effectiveness of MPM.