Spam detection in machine learning faces significant challenges from adversarial attacks, particularly evasion techniques, which manipulate models to bypass detection systems. These attacks highlight the need for more resilient classifiers capable of maintaining performance in adversarial conditions. This study evaluates several classifiers, including AdaBoost, Naive Bayes, SVM, RandomForest, and Decision Trees, using TF-IDF and Bag of Words (BOW) representations. AdaBoost and Naive Bayes achieved high performance with 99.44% accuracy, 99.44% \(F_1\) -Score, and 98.65% MCC using TF-IDF. Its robustness against two attacks was evaluated: DeepWordBug and a lightweight version of TextBugger, proposed in this paper for environments with limited resources. Despite its simplicity, this variant compromised models such as AdaBoost and, especially, Decision Trees, whose accuracy fell from approximately 98% to 40%. With DeepWordBug, SVM and RandomForest reduced their accuracy to 16.47% and 22.33%, respectively. Naive Bayes showed greater resilience, maintaining high accuracy and low ASR in both attacks. Additionally, to improve model interpretability and transparency, LIME was applied to provide insights into the decision-making process of these classifiers, aligning with Responsible AI principles.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Responsible AI in Adversarial Text Attacks: Evaluating the Security of Spam Detection

  • Samantha Acosta Ruiz,
  • Mireya Tovar Vidal,
  • José A. Reyes Ortiz

摘要

Spam detection in machine learning faces significant challenges from adversarial attacks, particularly evasion techniques, which manipulate models to bypass detection systems. These attacks highlight the need for more resilient classifiers capable of maintaining performance in adversarial conditions. This study evaluates several classifiers, including AdaBoost, Naive Bayes, SVM, RandomForest, and Decision Trees, using TF-IDF and Bag of Words (BOW) representations. AdaBoost and Naive Bayes achieved high performance with 99.44% accuracy, 99.44% \(F_1\) -Score, and 98.65% MCC using TF-IDF. Its robustness against two attacks was evaluated: DeepWordBug and a lightweight version of TextBugger, proposed in this paper for environments with limited resources. Despite its simplicity, this variant compromised models such as AdaBoost and, especially, Decision Trees, whose accuracy fell from approximately 98% to 40%. With DeepWordBug, SVM and RandomForest reduced their accuracy to 16.47% and 22.33%, respectively. Naive Bayes showed greater resilience, maintaining high accuracy and low ASR in both attacks. Additionally, to improve model interpretability and transparency, LIME was applied to provide insights into the decision-making process of these classifiers, aligning with Responsible AI principles.