Architectural Comparisons and Performance Evaluation of Transformer Models on Sequence Classification: A Case Study on Fake News Detection
摘要
The proliferation of fake news in the digital age poses a significant threat to information integrity and societal stability. This study addresses this critical challenge by presenting a comprehensive comparative analysis of seven diverse language models for fake news detection: LLAMA2, GPT-2, Mistral, RoBERTa, DistilBERT, GEMMA, and MAMBA. Using a carefully curated and balanced dataset, we evaluate these models based on accuracy, precision, recall, F1-score, and macro F1, while also considering their training efficiency and resource requirements. Our results reveal LLAMA2 as the top performer with 92.5% accuracy and 92.7% F1-score, closely followed by Mistral. We observe significant trade-offs between precision and recall across models, with Mistral excelling in precision (96.8%) and RoBERTa leading in recall (97.5%). Smaller models like GEMMA 2B show competitive performance, highlighting the impact of architectural innovations. This study offers valuable insights for selecting models based on various deployment scenarios, balancing performance with computational constraints. It also opens avenues for future research on optimizing AI-driven fake news detection systems, addressing the urgent need for effective tools to combat fake news.