STformer: Spatio-Temporal Transformer for Multivariate Time Series Anomaly Detection

Efficient anomaly detection and diagnosis in multivariate time series plays a critical role in modern industrial applications. The transformer-based model reveals the correlations among multivariate time series and their impact on various anomalies. However, the relationship of multivariate time series is inherently complex, and the existing models often fail to capture the intricate spatio-temporal dependency adequately. In order to effectively improve the model’s capabilities and obtain relevant information, we propose Spatio-Temporal Transformer (STformer), an innovative approach that explicitly leverages and exploits spatial and temporal dependency within multivariate time series. Firstly, based on the previous transformer model, we stack encoder layer blocks to reconstruct the time series for anomaly detection. Secondly, we divide the sequence into temporal patches and consider attention associations between patches. Specifically, we refer to this attention mechanism as Multi Head Patches Attention (MHPA), which enables comprehensive associations among the patches. Furthermore, we propose Spatio-Temporal Attention (STA), which exploits the correlation between spatial-temporal patches to capture spatiotemporal dependency. Extensive experiments on multivariate time series datasets including SMD, PSM, SMAP and MSL demonstrate the potential of spatio-temporal transformers in anomaly detection. The accuracy and F1 score achieved by STformer reach the state-of-the-art performance.