An emotion-driven, transformer-based network for multimodal fake news detection

Social media is filled with multimedia data in the form of news and is heavily impacting the daily lives of the people. However, the rise of fake news is causing distress and becoming a major source of concern. Several attempts have been made in the past to detect fake news, but it still remains a challenging problem. In this study, we propose an emotion-driven framework that extracts emotions from the multimodal data to identify fake news. We use the vision transformer, which removes the irrelevant data from the images and enhances the overall classification accuracy. To the best of our knowledge, this is the first work that incorporates multimodal emotions to detect fake news in the multimodal data, comprising of images and text. We conducted several experiments on five datasets: Twitter, Jruvika Fake News Dataset, Pontes Fake News Dataset, Risdal Fake News Dataset, and Fakeddit Multimodal Dataset, and evaluated the performance of the network by using Precision, Recall, F1 scores, Accuracy, and ROC curves. We also conducted an ablation study to verify the effectiveness of different components involved in the proposed architecture. The experimental results show that the proposed architecture outperforms the state-of-the-art and other baseline methods on all the evaluation metrics.