Misinformation Detection Through Multimodal Machine Learning Analysis of Eye-Tracking and Physiological Responses: A Proof-of-Concept Study
摘要
The digital age and social media have transformed news dissemination and consumption. Exposure to an overwhelming influx of fake news, together with content scheduling schemes tailored to amplify content consumption and sharing, hinders critical thinking and decision-making, leads to false memories, and fosters a pessimistic outlook on society. The last few years have seen a surge of interest in systems for the automated detection of fake news that can help counter the deleterious effects of fake news consumption. To this end, this study investigates how multimodal fusion of eye-tracking and peripheral physiology measures can contribute to automated fake news classification. Employing machine learning techniques, we compared single-modal (either physiology or eye-tracking) and multimodal (combined physiology and eye-tracking and self-assessment responses) features for news (fake/real) classification. The analysis reveals a better performance of multimodal over the single-modal classification of real vs. fake news, highlighting the advantage of integrating different behavioral metrics on detection systems based on readers’ cognitive/emotional responses. These findings contribute to the emerging field of multimodal interaction in news verification, specifically regarding the development of effective technical solutions for understanding and combating digital disinformation, as well as to the research of readers’ involvement and discernment when processing news content.