Towards a Holistic and Practical Adversarial Defense: A Modular Framework from Data to Model

To address the challenge of adversarial defense in image classification and object detection tasks, we propose a systematic adversarial defense framework based on modular design. This framework integrates multiple defense techniques, including adversarial patch removal, adversarial perturbation purification, multi-model ensemble, and output optimization strategies. It covers the entire defense process from the data level to the model level, and from the training stage to the inference stage, forming a comprehensive multi-level defense system. The core components of the system are the Fast Patch-Agnostic Defense (FAST-PAD), which leverages spatial heterogeneity for efficient and patch-agnostic protection, and the Multi-source Adversarial Perturbations Elimination (MAPE) module for purifying adversarial perturbations. Experimental results demonstrate that the proposed system effectively defends against both adversarial patches and perturbations. Built upon this systematic defense framework, we achieved the first place in the defense track of the National Adversarial Algorithm Challenge.