<p>Frequency-domain modeling has emerged as a powerful paradigm for time series forecasting, yet existing Transformer-based approaches face a fundamental challenge. While they effectively capture temporal dependencies, they cannot model the time-varying importance of frequency-domain features. To address this significant gap, we propose FCAFormer, a frequency-aware Transformer architecture that enhances frequency-domain modeling through two key innovations. First, our novel frequency-channel attention mechanism uniquely integrates multihead self-attention with frequency-adaptive channel attention, enabling the simultaneous learning of long-term temporal patterns and the dynamic importance of frequency features. Second, we introduce SparseCoreFFN, a computationally efficient feedforward network that employs strategic parameter sparsification to maintain model capacity while significantly reducing computational overhead. Extensive experiments across eight diverse real-world datasets demonstrate the effectiveness of FCAFormer: It achieves superior forecasting accuracy while effectively addressing the frequency-domain learning bias that has constrained previous approaches. Compared to existing baselines, FCAFormer consistently demonstrates notable improvements, offering a promising advancement in frequency-aware time series modeling.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

FCAFormer: multivariate time series forecasting combining channel attention and transformer in the frequency domain

  • Bin Xiao,
  • Zehao Ge,
  • Xingpeng Zhang,
  • Min Wang,
  • Zheng Chen,
  • Zhou Yang,
  • Yanxue Wu

摘要

Frequency-domain modeling has emerged as a powerful paradigm for time series forecasting, yet existing Transformer-based approaches face a fundamental challenge. While they effectively capture temporal dependencies, they cannot model the time-varying importance of frequency-domain features. To address this significant gap, we propose FCAFormer, a frequency-aware Transformer architecture that enhances frequency-domain modeling through two key innovations. First, our novel frequency-channel attention mechanism uniquely integrates multihead self-attention with frequency-adaptive channel attention, enabling the simultaneous learning of long-term temporal patterns and the dynamic importance of frequency features. Second, we introduce SparseCoreFFN, a computationally efficient feedforward network that employs strategic parameter sparsification to maintain model capacity while significantly reducing computational overhead. Extensive experiments across eight diverse real-world datasets demonstrate the effectiveness of FCAFormer: It achieves superior forecasting accuracy while effectively addressing the frequency-domain learning bias that has constrained previous approaches. Compared to existing baselines, FCAFormer consistently demonstrates notable improvements, offering a promising advancement in frequency-aware time series modeling.