Autocorrelation-guided temporal–spectral cross-attention integrated with deep residual probsparse transformers for hourly load forecasting

This paper presents a novel hybrid architecture for 24-h ahead electrical load forecasting by integrating signal decomposition, correlation-aware modeling, and deep ProbSparse attention mechanisms. First, the original load series is decomposed using optimized variational mode decomposition to extract informative frequency-aware components. A sequence of 24 Joint Temporal–Spectral Cross-Attention Blocks (JTS-CABs) forms the primary forecasting framework. Each JTS-CAB contains two Correlation-Aware Multi-Scale ProbSparse Attention Modules (CAMPAMs), operating in temporal and spectral domains, respectively. Each CAMPAM consists of fine-grained and coarse-grained branches for multi-scale representation learning. In the temporal branch, autocorrelation-guided attention groups highly self-correlated signals to enhance localized pattern learning, while the coarse-grained branch captures long-range dependencies. Cross-attention is further employed to fuse temporal and spectral representations. To refine preliminary forecasts, a deep residual ProbSparse transformer framework iteratively suppresses residual errors through sparse attention and residual refinement. Moreover, the ProbSparse mechanism significantly reduces computational complexity, improving scalability and suitability for real-time high-dimensional forecasting environments. Experimental results demonstrate superior performance over state-of-the-art methods, achieving up to 8.9% lower MAPE under noisy and nonstationary conditions.