Enhancing Fast Adversarial Training via Adaptive Self-knowledge Dynamic Guidance
摘要
Adversarial training (AT) significantly improves adversarial robustness, but generating adversarial examples (AEs) is costly. Fast adversarial training (FAT) reduces the cost, but faces the catastrophic overfitting (CO) problem. Existing solutions adopt a fixed strategy, hindering the model from adapting to robustness differences due to example changes. Thus, an adaptive self-knowledge dynamic guidance FAT algorithm, FGSM-ASKDG, is proposed from the perspective of example changes. First, the adversarial initialization strategy of the gradient momentum perturbation is dynamically adjusted based on the degree of robustness of the model to generate high-quality AEs. Second, the cross-entropy loss of abnormal AEs and the dynamically guided regularization term are introduced to penalize the abnormal AEs and mitigate training instability. Third, the degree of label relaxation is dynamically adjusted in accuracy based on the initialized examples to balance internal and external optimization. Experiments on three datasets and three backbones show that the proposed method effectively prevents CO, remarkably improves adversarial robustness, and is more than three times computationally efficient than the multi-step AT algorithm.