MTC-VAD: cross-modal temporal coherence modeling for video anomaly detection
摘要
Video anomaly detection seeks to automatically identify abnormal events deviating from normal video sequences, holding significant promise for public security, traffic monitoring and autonomous driving applications. Current approaches however face substantial challenges when processing multi-source heterogeneous data, particularly regarding inadequate temporal relationship modeling and inherent modality asynchrony coupled with feature heterogeneity. This paper presents a novel video anomaly detection framework grounded in cross-modal temporal coherence modeling. Firstly, we addresses a multi-modal feature distribution discrepancies through cross-modal feature distribution alignment module that first establishes a reference space using CLIP-encoded video features, then achieves hierarchical alignment via learnable affine transformations for cross-modal dynamic range adaptation, and finally conducts asymmetric distribution calibration combining Wasserstein-distance-based core alignment with exponentially-constrained outlier compression. Subsequently, a temporal-aware hybrid attention fusion mechanism enables efficient multi-modal interaction through spatial replacement strategies and Monte Carlo modality sampling. Lastly, we introduce a cross-modal temporal self-attention module that quantifies cross-modal consistency between segments alongside a discrepancy maximization fusion network to sharpen normal/abnormal pattern discrimination. Comprehensive evaluations on UCF-Crime, XD-Violence and ShanghaiTech datasets demonstrate the method's effectiveness, achieving respective scores of 87.32% (AUC), 80.58% (AP) and 97.16% (AUC), confirming its superior anomaly detection capabilities.