Learning to route in time and frequency domains: a dual-domain MoE transformer for multi-horizon forecasting
摘要
Accurate long-term electrical load forecasting is required for reliable smart grid operation, yet it remains difficult due to multi-scale periodic patterns and non-stationary temporal variations across different prediction horizons. This paper presents MoE-Transformer, a dual-domain forecasting framework that learns to route representations in both the time and frequency domains through reinforcement learning. To mitigate spectral misalignment in multi-step forecasting, we introduce an Extended Discrete Fourier Transform (Extended DFT) that aligns the input spectrum with the frequency grid of the full prediction window. The proposed model incorporates parallel Mixture-of-Experts modules in the time and frequency domains (T-MoE and F-MoE), where domain-specific experts capture complementary temporal dynamics and spectral structures. Expert routing in each domain is modeled as an independent Markov Decision Process and optimized using reinforcement learning to jointly consider forecasting accuracy, routing consistency, and balanced expert utilization. Experiments on five benchmark datasets, including ETTh1, Electricity, and Traffic, across four forecasting horizons show that MoE-Transformer achieves MSE reductions of 50.9–56.9% relative to state-of-the-art baselines under matched training protocols. Relative to a same-capacity dense Transformer baseline on NVIDIA RTX 4090, sparse top-1 expert activation reduces peak GPU memory by