LCformer: Linear Convolutional Decomposed Transformer for Long-Term Series Forecasting

Transformer-based methods have shown excellent results in long-term series forecasting, but they still suffer from high time and space costs; difficulties in analysing sequence correlation due to entanglement of the original sequence; bottleneck in information utilisation due to the dot-product pattern of the attention mechanism. To address these problems, we propose a sequence decomposition architecture to identify the different features of sub-series decomposed from the original time series. We then utilize causal convolution to solve the information bottleneck problem caused by the attention mechanism’s dot-product pattern. To further improve the efficiency of the model in handling long-term series forecasting, we propose the Linear Convolution Transformer (LCformer) based on a linear self-attention mechanism with O(n) complexity, which exhibits superior prediction performance and lower consumption on long-term series prediction problems. Experimental results on two different types of benchmark datasets show that the LCformer exhibits better prediction performance compared to those of the state-of-the-art Transformer-based methods, and exhibits near linear complexity for long series prediction.