Fusion of Image Representations for Time Series Classification with Deep Learning

Electronics and sensors are a rich source of continuous data. These devices generate unbounded and ordered data sequences at a high-frequency rate (e.g., 100 Hz) called time series. Time series can represent the data of a patient’s heartbeats to the seismic waves caused by the movement of materials within the Earth. Discovering and extracting patterns from time series is essential for supervised machine learning applications such as identifying cardiac arrhythmia or earthquake prediction. However, in many cases, it is challenging to identify hidden sequential patterns when these data are represented in their primitive form in the time domain. In order to reveal hidden patterns, a typical approach is to change the data representation by transforms (e.g., from time to frequency domain). In this paper, we propose a two-dimensional representation named Fusion of Image Representations for Time Series (FIRTS) to represent time series as images that reveal hidden patterns, improving the accuracy of supervised classifiers. Transforming signals to images allows using accurate deep learning methods developed and optimized for image classification (e.g., object recognition) in problems that initially generate one-dimensional data. FIRTS is obtained from the simple and efficient fusion of weak representations for time series. In a comprehensive experimental evaluation with 100 datasets, five representations, and six Convolutional Neural Networks (CNN) architectures, we demonstrate that FIRTS statistically outperforms distance-based methods and CNN models using other representations or in time domain.