Sparse Attention-Based Imputation Network for Time Series
摘要
Time series data related to traffic and air quality are useful indicators for urban planning but they frequently have missing values. Imputation of multi-sensor time series data is thus a vital pre-processing step for forecasting, anomaly detection and other downstream tasks. We develop an efficient architecture – Sparse Attention-based Imputation Network for Time series (SAINT) – which outperforms the state-of-the-art imputation networks. Efficiency is achieved by separating the computations on the space-time product graph sequentially into channel independent temporal attention and sparse space-time transformer. This channel independent network can reduce overfitting to effectively represent general temporal patterns. Sparse space-time transformer performs message passing on the spatial graph conditioned on time. We consider real-world datasets for evaluation – PEMS-BAY, METR-LA and AQI – which are gathered from sensor networks in major cities. We demonstrate the effectiveness and robustness of SAINT across complex missing data scenarios. Additionally, SAINT generalizes well to short-term forecasting and is practical for long-term forecasting with limited resources.