PAPT: Periodic-Aware Transformer with Polynomial Trend Fitting for General Time Series Forecasting
摘要
Time series forecasting has been widely applied across multiple domains including databases, finance and transportation. While Transformer-based models have made significant progress in capturing long-term dependencies through attention mechanisms, they struggle with modeling mixed multi-scale periodicity and dynamic trends. In this study, we propose PAPT, a Transformer-based general framework to capture multi-periodicity and long-term trends in time series. On one hand, we introduce Periodic Attention Positional Encoding (PAPE), which incorporates multiple periodic patterns into the attention encoding to enhance weight correlations at periodic synchronization points in the attention matrix. Furthermore, we employ a two-layer feature generation module (TGM) to capture fluctuations within cycles. On the other hand, we design an adaptive polynomial basis trend fitting module (PTM) that learns trend features through weight adjustment and fits them with multiple distinct polynomial bases. Compared with 8 baselines, PAPT demonstrates state-of-the-art forecasting performance through extensive experiments conducted on 10 real-world datasets in 4 domains. The code and datasets are shown in https://github.com/wxy1225/Periodic-Attention-with-Polynomial-Trend-Fitting-Transformer.git .