In recent years, teacher-student distillation has become a prevalent approach for image anomaly detection in industrial and medical scenarios. However, current state-of-the-art methods often prioritize performance while overlooking the increased deployment cost introduced by asymmetric teacher-student architectures in practical applications. To address this issue, we propose a distillation framework based on vanilla Vision Transformers (ViT), which bridges the performance gap between symmetric and asymmetric teacher-student architectures. To mitigate the mimicry issue caused by directly applying symmetric ViT models for distillation, we design Anomaly Synthesis Module (ASM) and Dynamic Feature Selection Module (DFSM), which reduce the similarity between anomaly-related features extracted by the student and teacher network. Experiments conducted on five datasets, including MVTec AD, VisA, and BTAD, demonstrate that our method not only outperforms prior symmetric distillation models but also achieves superior anomaly localization compared to asymmetric teacher-student architectures.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Revisiting Symmetric Teacher-Student Network Distillation for Anomaly Detection

  • Qunyi Zhang,
  • Jiaqi Liu,
  • Guoyang Xie,
  • Liewen Liao,
  • Yongming Chen,
  • Xiaoning Lei,
  • Annan Shu,
  • Guannan Jiang,
  • Songan Zhang

摘要

In recent years, teacher-student distillation has become a prevalent approach for image anomaly detection in industrial and medical scenarios. However, current state-of-the-art methods often prioritize performance while overlooking the increased deployment cost introduced by asymmetric teacher-student architectures in practical applications. To address this issue, we propose a distillation framework based on vanilla Vision Transformers (ViT), which bridges the performance gap between symmetric and asymmetric teacher-student architectures. To mitigate the mimicry issue caused by directly applying symmetric ViT models for distillation, we design Anomaly Synthesis Module (ASM) and Dynamic Feature Selection Module (DFSM), which reduce the similarity between anomaly-related features extracted by the student and teacher network. Experiments conducted on five datasets, including MVTec AD, VisA, and BTAD, demonstrate that our method not only outperforms prior symmetric distillation models but also achieves superior anomaly localization compared to asymmetric teacher-student architectures.