Deep learning (DL) has been poised to become an integral part of many critical systems. Despite its robustness to natural variations, it is highly susceptible to adversarial examples (AEs) generated by subtle, imperceptible perturbations. As a standard defense form, Adversarial Training (AT) enhances robustness by incorporating AEs during training, yet the distributional gap between clean examples and AEs often compromises standard accuracy, posing a trade-off challenge. By routing examples to matching experts, the Mixture of Experts (MoE) offers a solution. However, current MoE-based AT methods are limited by global pooling-based routers and fail to exploit routing capabilities fully during the AT. This work proposes DRAMoE (Dynamic Router-guided Adversarial Mixture of Experts), a novel MoE architecture integrating AT with a divide-and-conquer strategy. DRAMoE comprises two core components: a Dynamic Channel Attention Router (DCAR) and Router-Guided Adversarial Training (RGAT). DCAR enhances feature representation, while RGAT optimizes AEs generation and designs diverse losses. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet demonstrate that DRAMoE improves robust accuracy over the baseline while maintaining competitive standard accuracy.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

DRAMoE: Boosting Adversarial Robustness with Adversarial Training and Adaptive Mixture of Experts

  • Yu Fu,
  • Hengzhi Xie,
  • Ming Yang,
  • Denghui Zhang

摘要

Deep learning (DL) has been poised to become an integral part of many critical systems. Despite its robustness to natural variations, it is highly susceptible to adversarial examples (AEs) generated by subtle, imperceptible perturbations. As a standard defense form, Adversarial Training (AT) enhances robustness by incorporating AEs during training, yet the distributional gap between clean examples and AEs often compromises standard accuracy, posing a trade-off challenge. By routing examples to matching experts, the Mixture of Experts (MoE) offers a solution. However, current MoE-based AT methods are limited by global pooling-based routers and fail to exploit routing capabilities fully during the AT. This work proposes DRAMoE (Dynamic Router-guided Adversarial Mixture of Experts), a novel MoE architecture integrating AT with a divide-and-conquer strategy. DRAMoE comprises two core components: a Dynamic Channel Attention Router (DCAR) and Router-Guided Adversarial Training (RGAT). DCAR enhances feature representation, while RGAT optimizes AEs generation and designs diverse losses. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet demonstrate that DRAMoE improves robust accuracy over the baseline while maintaining competitive standard accuracy.