Robust Adversarial Defense: An Analysis on Use of Auto-Inpainting

In recent years, adversarial patch attacks have become a major concern since they can seriously compromise the security and reliability of deep neural networks. These attacks involve modifying a clean image by adding a patch with carefully crafted pixels, which can confuse the network and cause it to misclassify the image. Our novel approach addresses this problem by removing the patch from the image and restoring the missing pixels using various image inpainting techniques. We automatically detect the adversarial patch using a method called Fast Score Class Activation Map which localizes the salient region in an image. Further, we replace the patch with inpainted pixels that utilize the neighbourhing pixels and a local or a global context in the image. This approach ensures the integrity of the image’s original structure while effectively tackling the adversarial nature of the patch, regardless of whether it is universal, robust, or targeted. Importantly, our method works even in the black-box setting without assuming specific details about the patch such as its location, shape or size, making it suitable for deployment. Moreover, our approach does not necessitate retraining the neural network on adversarial examples, thus, adding to its practicality in real-world scenarios. Our experiments show that adversarial patch attacks can reduce classification accuracy on the ImageNet100 dataset from 96.78% to as low as 0.01%. The proposed black-box method recovers classification accuracy to 80.89%, while the best-performing white-box method reaches only 76.80%. Furthermore, the defense is oblivious to the input being adversarial or benign and achieves an accuracy of 84.20% on clean images demonstrating its effectiveness in either case. This defense is especially relevant for real-world applications, such as autonomous vehicles and surveillance systems, where adversarial patches can compromise critical decisions by misleading object detection and classification.