An information retrieval language model-based zero- and few-shot learning for propaganda detection in social media content
摘要
Multitask learning strengthens information retrieval (IR) by enabling systems to address multiple tasks, making it valuable for detecting propaganda in social media content. Although large language models (LLMs) advance IR through zero- and few-shot learning with minimal labeled data, challenges remain in capturing subtle contextual cues and managing the computational cost of large-scale content moderation. This study proposes an information retrieval language model (IRLM), a multistage, fine-tuned open IR framework for detecting propaganda in news articles shared on social media. The framework builds on LLMs capable of handling non-sequential word patterns and long-range dependencies. Contextual representations from transformer-based encoders (e.g., BERT/RoBERTa) are pooled and fed into a span-level propaganda classifier, and the model is evaluated on the propaganda text corpus (PTC) and propaganda text (ProText). The results highlight both the potential and the limitations of LLM-based IR models. Standard binary classifiers struggle to generalize and often associate labels with specific sources rather than content, underscoring the need for more robust training data. Incorporating maximum-entropy classifiers with zero-shot features improves accuracy without task-specific fine-tuning, while adding IRLM’s zero- and few-shot representations further boosts performance on PTC and ProText. LLMs such as GPT-3 and BERT capture complex patterns and generalize more effectively than simpler baselines. Across experiments, LLM-based variants achieve high F1-scores with about 60% fewer training iterations, reducing training cost while maintaining strong predictive performance. Overall, the proposed scalable framework offers a practical tool for propaganda detection and trustworthy digital communication.