A Note on Linear Time Series Prediction

We consider the problem of univariate time series prediction from an elementary machine learning point of view. Beginning with the question of whether and how Principal Component Analysis (PCA) can be used for time series prediction, we describe a simple methodology and attempt to classify PCA-based prediction in terms of statistics, signal processing and dynamical systems theory. Moreover, we extend the unsupervised scenario to a self-supervised linear regression scenario and develop a unifying perspective. In this regard, we review several related techniques, namely autoregressive (AR) and moving-average (MA) models, Singular Spectrum Analysis (SSA), Wiener filtering, and the discrete Fourier transform (DFT). By presenting these methods in a unified way, we can show how PCA-based time series prediction can be categorized in different settings of stochastic and deterministic models. Finally, we show the distinct relation between PCA-based prediction and (finite-order) MA processes and propose a refined methodology.