Nearest Memory Augmented Feature Reconstruction for Unified Anomaly Detection

Reconstruction-based anomaly detection methods expect to reconstruct normality well but fail for abnormality. Memory modules have been exploited to avoid reconstructing anomalies, but they may overgeneralize by using memory in a weighted manner. Additionally, existing methods often require separate models for different objects. In this work, we propose nearest memory augmented feature reconstruction for unified anomaly detection. Specifically, the novel nearest memory addressing (NMA) module enables memory items to record normal prototypical patterns individually. In this way, the risk of over-generalization is mitigated while the capacity of the memory item is fully exploited. To overcome the constraint of training caused by NMA that has no real gradient defined, we perform end-to-end training with straight-through gradient estimation and exponential moving average. Moreover, we introduce the feature reconstruction paradigm to avoid the reconstruction challenge in the image space caused by information loss of the memory mechanism. As a result, our method can unify anomaly detection for multiple categories. Extensive experiments show that our method achieves state-of-the-art performance on MVTecAD dataset under the unified setting. Remarkably, it achieves comparable or better performance than other algorithms under the separate setting.