Machine Learning-Based Detection and Categorization of Malicious Accounts on Social Media

In recent years, Online Social Networking (OSN) platforms have become an integral component of people’s lives. The availability of the massive amount of information generated on these platforms along with their open nature attracts cybercriminals who create fake human accounts or bots with an intention of spamming, scamming, disseminating hate speech or disinformation, and more. Thus, automatically detecting such malicious accounts is an important problem that we address in this paper. We design a machine learning model to classify an X (formerly Twitter) account into one of the following categories—genuine accounts, social spambots, traditional spambots, and fake followers, with further classification into subcategories for social and traditional spambots. We use tweets made by a user, tweet-based features, and user-based features to train several machine learning classifiers. Our results demonstrate that the DistilBERT model shows the best performance among all the models by achieving an accuracy of around 91%. We create a web application that accepts a link to a user account, uses the Twitter API to pull the user’s public data, and uses the DistilBERT model to classify it into a category. This paper presents the results of our preliminary investigation and lays the groundwork for further detailed analysis for malicious account detection.