Learning dual updatable memory modules for video anomaly detection

We propose a novel video anomaly detection method that leverages two updatable memory modules to learn and update prototypical patterns of normal and abnormal data within an autoencoder (AE) framework. To enhance the robustness of the model, we employ a pseudo anomaly synthesizer to generate synthetic anomalies from normal data, and train the AE to minimize the reconstruction loss on pseudo anomalies while maximizing it on normal data. The memory modules are optimized using a feature compactness loss and a separateness loss to refine the representation of details, and skip connections are incorporated to prevent the recording of only the most prototypical patterns. Additionally, a memory loss is proposed to enhance the distinction between the two memory modules, thereby enabling effective anomaly detection. Experimental results demonstrate the efficacy of our approach, underscoring the importance of the two updatable memory modules in achieving state-of-the-art performance in video anomaly detection. Our code is available at https://github.com/SVIL2024/Memup.git.