Transformer-Based Models for Detecting Bots on Twitter

The rise of social media platforms like Twitter has led to an increased presence of automated accounts, commonly referred to as bots, which often spread misinformation, engage in spam, and manipulate online discourse. Detecting these bots is crucial to maintaining the integrity of online conversations and user trust. In recent years, advanced Natural Language Processing (NLP) techniques, particularly Transformer-based models, have demonstrated remarkable capabilities in various language-related tasks. This paper investigates the effectiveness of employing Transformer-based models, with a specific focus on BERT (Bidirectional Encoder Representations from Transformers), for detecting bots on Twitter. Our methodology involves preprocessing Twitter data, generating contextual embeddings using BERT, and designing classification models that learn to differentiate between human users and bots. Experiments were carried out adopting advanced Language Models to construct an encoding of the tweet to create a potential input vector on top of BERT and its variants. By employing Transformer-based models, we achieve significant improvements in bot detection accuracy compared to traditional methods such as Word2Vec and Glove.