Being a full-blown dissemination of false news at the hands of social media has become a serious threat: it confuses public understanding and creates mistrust in society. In this research we have approached the problem of detecting fake news on a multimodal level, which means that both textual and visual misinformation have to be jointly scrutinized for a more robust classification. We propose the BAFT (BERT Attention Fusion Transformer) model to deal with this: a hybrid deep learning architecture-wherein BERT builds a textual encoder, ResNet50 extracts image features, multi-head attention learns contextual information, and transformer encoders perform sequence modeling-all fused after optimization for joint-multimodal learning. The model is trained and evaluated with the Fakeddit dataset of real-world social media posts containing text and image, scattered in six fine-grained categories. Experimental results show that BAFT ranks among the highest with an accuracy of 97.58%, surpassing almost all standards in multimodal detection. Moreover, with the advent of XAI techniques, interpretability has been injected into the decision-making process of the model, marking an essential step forward toward transparency and trust. The method is best suited to be implemented for monitoring and detecting misinformation on social media. The future work will address real-time implementation and speeding up for maximized applicability.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi-modal Fake News Detection on Online Social Media Using Machine Learning and Explainable AI

  • Khushi Jain,
  • Arunima Jaiswal,
  • Reena,
  • Jhalak Chahar,
  • Smita Maurya,
  • Nitin Sachdeva

摘要

Being a full-blown dissemination of false news at the hands of social media has become a serious threat: it confuses public understanding and creates mistrust in society. In this research we have approached the problem of detecting fake news on a multimodal level, which means that both textual and visual misinformation have to be jointly scrutinized for a more robust classification. We propose the BAFT (BERT Attention Fusion Transformer) model to deal with this: a hybrid deep learning architecture-wherein BERT builds a textual encoder, ResNet50 extracts image features, multi-head attention learns contextual information, and transformer encoders perform sequence modeling-all fused after optimization for joint-multimodal learning. The model is trained and evaluated with the Fakeddit dataset of real-world social media posts containing text and image, scattered in six fine-grained categories. Experimental results show that BAFT ranks among the highest with an accuracy of 97.58%, surpassing almost all standards in multimodal detection. Moreover, with the advent of XAI techniques, interpretability has been injected into the decision-making process of the model, marking an essential step forward toward transparency and trust. The method is best suited to be implemented for monitoring and detecting misinformation on social media. The future work will address real-time implementation and speeding up for maximized applicability.