Attention-based misaligned spatiotemporal auto-encoder for video anomaly detection

To face the shortcomings of the auto-encoder (AE) algorithm in terms of weak dynamic object extraction and strong reconstruction ability in abnormal situations for video anomaly detection, an attention-based misaligned spatiotemporal AE (AMS-AE) model is proposed for video anomaly detection. The AMS-AE model uses a pseudo-abnormal generation module (PAGM) to dislocate the spatiotemporal information of the video and then uses an attention mechanism algorithm to selectively capture the foreground regions. Finally, it reconstructs the video frames through encoding and decoding. During the training process, normal data is used for training, while the abnormal frames generated by the PAGM are introduced as anomalies to limit the reconstruction ability of the auto-encoding algorithm on abnormal data, which facilitates better detection of anomalies during testing phase. The experimental results show that the AMS-AE algorithm achieves significantly better performance than some other algorithms on three public datasets (UCSD Ped2, CUHK Avenue, and Shanghai Tech), effectively improving the accuracy and robustness of video anomaly detection. The area under the ROC curve (AUC) values of 98.7%, 89.7%, and 74.0% were achieved on the above public datasets, respectively.