<p>The time-dependent and long sequential time-series is always considered as a challenging problem. It normally demands the robust machine learning (ML) or deep learning (DL)-based techniques to efficiently preserve the long-ranged dependencies within intricate time-series datasets. In recent years, transformer-based architectures have emerged as a powerful tool across various data mining fields, particularly in natural language processing (NLP). These architectures are widely regarded for their ability to effectively handle long-range dependencies in sequence-based data structures, such as text and time-dependent datasets. Consequently, transformers have paved the way for advancements in complex long-sequence time-series forecasting tasks. However, recent studies have highlighted limitations in transformer-based predictive models, particularly in their ability to fully capture the joint spatial and temporal representations within input sequences. To address these challenges, we propose a novel approach in this paper, multi-channel attention transformer for time-series (MAT4TS). Our MAT4TS model enhances the performance of long-sequence time-series forecasting by extending the conventional self-attention mechanism. This enhanced self-attention is complemented by additional multi-channel filtering and embedding processes, leveraging convolutional neural network (CNN)-based architectures. By integrating these processes, our model is able to extract richer spatial–temporal information from long-range sequences, ultimately leading to more accurate prediction outcomes. The comprehensive comparative studies within real-world long-sequential time-series datasets demonstrated the effectiveness as well as superiority of our proposed MAT4TS model for dealing with long-ranged time-series prediction problem in comparing with previous state-of-the-art baselines.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A novel approach of multi-channel attention mechanism for long-sequential multivariate time-series prediction problem

  • Tham Vo,
  • Linh Nguyen Thi My

摘要

The time-dependent and long sequential time-series is always considered as a challenging problem. It normally demands the robust machine learning (ML) or deep learning (DL)-based techniques to efficiently preserve the long-ranged dependencies within intricate time-series datasets. In recent years, transformer-based architectures have emerged as a powerful tool across various data mining fields, particularly in natural language processing (NLP). These architectures are widely regarded for their ability to effectively handle long-range dependencies in sequence-based data structures, such as text and time-dependent datasets. Consequently, transformers have paved the way for advancements in complex long-sequence time-series forecasting tasks. However, recent studies have highlighted limitations in transformer-based predictive models, particularly in their ability to fully capture the joint spatial and temporal representations within input sequences. To address these challenges, we propose a novel approach in this paper, multi-channel attention transformer for time-series (MAT4TS). Our MAT4TS model enhances the performance of long-sequence time-series forecasting by extending the conventional self-attention mechanism. This enhanced self-attention is complemented by additional multi-channel filtering and embedding processes, leveraging convolutional neural network (CNN)-based architectures. By integrating these processes, our model is able to extract richer spatial–temporal information from long-range sequences, ultimately leading to more accurate prediction outcomes. The comprehensive comparative studies within real-world long-sequential time-series datasets demonstrated the effectiveness as well as superiority of our proposed MAT4TS model for dealing with long-ranged time-series prediction problem in comparing with previous state-of-the-art baselines.