Masked feature reconstruction distillation for unsupervised anomaly detection

Knowledge distillation based approaches have achieved decent performance in addressing unsupervised anomaly localization problems where exists no negative samples during training phase. However, most of them encouraged student to directly mimic same outputs as teacher with same normal input image, by which student learn no information about differentiating abnormal features with normal ones. And the similar student-teacher structure will cause a over-generalized problem. In this article, we propose a Feature Reconstruction Distillation strategy to transfer the aim of student network from directly mimic teacher to reconstruct masked features to normal with the guidance of teacher. By simply random mask pixels in multiple scale feature maps in a certain proportion, over-generalized regions generated by symmetrical student-teacher encoder architecture will be eliminated and pseudo defects which have a stronger representation than previous Image Painting simulation are introduced. Then a Trapezoidal Feature Reconstruction module is proposed to reconstruct masked features, which can localizes defects of multiple scales adaptively and retain useful information for normal reconstruction. Extensive experiments on MVTec AD dataset demonstrates that our method achieves significant improvement compared to baseline and is competitive with other recent methods, \(98.70\%\) $98.70\%$ in detection-level AUC, \(98.30\%\) $98.30\%$ and \(95.12\%\) $95.12\%$ in pixel-level AUC and PRO-AUC.