Online Conversation-Based Social Engineering Detection Using Machine Learning

Social engineering through online conversations can occur via phone calls, Skype, or Google Meet, among others. This paper presents a machine learning-based classifier for detecting scam conversations in various online formats, including live call conversations. However, selecting an appropriate dataset and the optimal vectorization technique for the algorithm remains challenging, and many fraudulent scams remain undetectable in online conversations. Consequently, six experiments were conducted to apply a machine learning classifier, resulting in 108 outcomes. All six experiments demonstrate that different classifiers possess unique strengths and weaknesses when applied to different scenarios. When compared to Doc2Vec, the vectorization techniques of Universal Sentence Encoder yield excellent results. Among various clustering methods, K-Means and the EM algorithm perform exceptionally well. The results reveal that Random Forest and CatBoost classifiers outperform others in terms of accuracy, precision, recall, and F1-score across all cases. These findings can contribute to enhancing the detection of scam attempts in live call conversations, thus helping protect individuals from falling victim to scams.