FCAFormer: multivariate time series forecasting combining channel attention and transformer in the frequency domain
摘要
Frequency-domain modeling has emerged as a powerful paradigm for time series forecasting, yet existing Transformer-based approaches face a fundamental challenge. While they effectively capture temporal dependencies, they cannot model the time-varying importance of frequency-domain features. To address this significant gap, we propose FCAFormer, a frequency-aware Transformer architecture that enhances frequency-domain modeling through two key innovations. First, our novel frequency-channel attention mechanism uniquely integrates multihead self-attention with frequency-adaptive channel attention, enabling the simultaneous learning of long-term temporal patterns and the dynamic importance of frequency features. Second, we introduce SparseCoreFFN, a computationally efficient feedforward network that employs strategic parameter sparsification to maintain model capacity while significantly reducing computational overhead. Extensive experiments across eight diverse real-world datasets demonstrate the effectiveness of FCAFormer: It achieves superior forecasting accuracy while effectively addressing the frequency-domain learning bias that has constrained previous approaches. Compared to existing baselines, FCAFormer consistently demonstrates notable improvements, offering a promising advancement in frequency-aware time series modeling.