Multivariate time series (MTS) forecasting plays a critical role across various domains. Recent advancements in deep learning, particularly transformer-based methods, have shown promising results. These methods partition time series into patches and treat them as tokens to capture complex temporal patterns. However, two key challenges remain: effectively modeling multi-scale temporal dependencies and accurately capturing inter-variable relationships. Existing models typically adopt a single-scale patch division strategy, overlooking the inherently multi-scale nature of time series, which is crucial for precise forecasting. Additionally, these models either fully mix all channels or process them independently, failing to explicitly emphasize inter-variable dependencies. To address these limitations, we propose the Multi-scale Perception Dual Attention Transformer (DTMP). DTMP leverages periodic information to segment time series into multiple scales and adaptively selects the most suitable scale for each instance. To enhance temporal modeling, we introduce a dual attention mechanism that separately captures global and local dependencies. Furthermore, given that inter-variable dependencies may vary across different scales, we incorporate graph structure learning at each scale and employ graph convolution to model scale-specific variable interactions. Extensive experiments on multiple real-world datasets demonstrate the superior forecasting performance of DTMP, highlighting its effectiveness in capturing both multi-scale temporal dependencies and inter-variable relationships.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dual Attention Transformer with Multi-scale Perception for Multivariate Time Series Forecasting

  • Fengjie Li,
  • Wu Peng,
  • Mingyu Zhang,
  • Miao Wang,
  • Hong Zhang

摘要

Multivariate time series (MTS) forecasting plays a critical role across various domains. Recent advancements in deep learning, particularly transformer-based methods, have shown promising results. These methods partition time series into patches and treat them as tokens to capture complex temporal patterns. However, two key challenges remain: effectively modeling multi-scale temporal dependencies and accurately capturing inter-variable relationships. Existing models typically adopt a single-scale patch division strategy, overlooking the inherently multi-scale nature of time series, which is crucial for precise forecasting. Additionally, these models either fully mix all channels or process them independently, failing to explicitly emphasize inter-variable dependencies. To address these limitations, we propose the Multi-scale Perception Dual Attention Transformer (DTMP). DTMP leverages periodic information to segment time series into multiple scales and adaptively selects the most suitable scale for each instance. To enhance temporal modeling, we introduce a dual attention mechanism that separately captures global and local dependencies. Furthermore, given that inter-variable dependencies may vary across different scales, we incorporate graph structure learning at each scale and employ graph convolution to model scale-specific variable interactions. Extensive experiments on multiple real-world datasets demonstrate the superior forecasting performance of DTMP, highlighting its effectiveness in capturing both multi-scale temporal dependencies and inter-variable relationships.