Evaluating Deep Learning for Cross-Domains Fake News Detection

With the rise of social media users, the quick transmission of news without sufficient verification has become a common problem. The proliferation of fake news across various social media platforms poses enormous harm to society and affects the news industry’s credibility. Therefore, it is critical to develop effective automated algorithms to detect deceptive articles. We show that existing models for fake news detection based on deep learning have limitations in terms of generalizability when confronted with a variety of news sources. Current deep learning models frequently fail to generalize adequately across different datasets, resulting in inferior performance. In this paper, we investigate the performance of numerous deep learning models on multiple fake news datasets, each with distinct characteristics. Our goal is to assess these models’ performance within the same dataset and across other datasets in the domain of fake news. We aim to acquire useful insights into the models’ robustness and generalizability across multiple datasets. We carried out an extensive set of experiments with five deep-learning models and seven datasets. These models are tested within a domain and across domains, i.e., on the same domain where they are trained and on other domains that were not seen during training. Our results show that these models cannot be generalized over various datasets and domains. The results reveal that these models exhibit high accuracy (around 99%) when tested on the dataset they were trained on but they experience a significant drop in performance (around 30%) when evaluated on different datasets.