Afcc: automatic fact-checkers’ consensus and credibility assessment for fake news detection

In today’s digital age, the importance of fact-checking is paramount as misinformation, disinformation, and fake news proliferate across online social networks (OSN), posing serious societal risks. However, a significant challenge in fact-checking is the lack of standardization and consistency in the rating labels used by fact-checkers, which can confuse the public. Furthermore, fact-checkers’ credibility can fluctuate due to various factors, potentially influencing the public’s confidence in their verdicts. Despite numerous efforts to explore the foundations of fact-checking in combating fake news, the automation of consensus-building among fact-checkers based on their credibility has not been previously addressed. The Automatic Fact-Checkers’ Consensus and Credibility Assessment (AFCC) system introduces a groundbreaking solution to this issue. It is designed to shift from a variety of textual rating labels to a standardized numerical rating system for fact-checked news and claims, thereby facilitating an automated process for achieving consensus and incorporating an innovative module for adjusting the credibility of fact-checkers to mitigate biases and inconsistencies. This methodology is aimed at reducing the impact of divergent ratings by employing weighted credibility assessments, ensuring a fair and accurate representation of fact-checkers’ credibility. The effectiveness of the AFCC system was verified using synthetic data due to the challenges associated with employing real-world datasets, which often lack multiple textual ratings for the same claim and necessary credibility scores. This research is critical for fact-checkers, media entities, social media platforms, journalists, and scholars engaged in the study of information disorder, providing a structured method to improve the reliability and standardization of the fact-checking process.