Extending Minimum Prediction Deviation as a Defence Against Adversarial Attacks

Machine learning can detect many types of cybersecurity attacks, adding a layer of security to various systems. However, machine learning is easily compromised by adversarial attacks, such as evasion attacks, model stealing, and data poisoning. The existence of adversarial machine learning raises the requirement for a robust defence. While many defences have been proposed, most have been broken shortly thereafter. Further, most defences are domain-specific and designed for a specific type of dataset or task. There is a need to find a suitable defence that prevents machine learning models from being exploited (Carlin et al. in On evaluating adversarial robustness, [1]). We build on previous work utilizing the Minimum Prediction Deviation (MPD), an uncertainty metric, to detect adversarial examples. Previously, MPD was shown to be very effective on a tabular phishing dataset (Santhosh et al. in AI 2022: advances in artificial intelligence: 35th Australasian joint conference, AI 2022, Springer, pp. 719–732, [2]). However, it was unknown if it would extend to other domains such as vision, malware, IDS, and non-binary classification tasks. In this work, we demonstrate that MPD can be used on multiple domains (IDS, malware, XSS, and computer vision), in a multi-class classification setting and with gradient and gradient-free attacks to detect adversarial examples. We demonstrate that MPD effectively detects 93% adversarial samples across all datasets (heterogeneous and homogeneous), with an average false positive of 2%. This approach showcases an effective defence mechanism to mitigate the issue of compromised machine learning, thus enhancing the security and reliability of various systems.