Large Scale Gaussian Processes with Matheron’s Update Rule and Karhunen-Loève Expansion

Gaussian processes have become essential for nonparametric function estimation and are widely used in many fields, like machine learning. In this paper, large scale Gaussian process regression (GPR) is investigated. This problem is related to the simulation of high-dimensional Gaussian vectors truncated on the intersection of a set of hyperplanes. The main idea is to combine both Matheron’s update rule (MUR) and Karhunen-Loève expansion (KLE). First, by the MUR we show that simulating from the posterior distribution can be achieved without computing the posterior covariance matrix and its decomposition. Second, by splitting the input domain into smallest nonoverlapping subdomains, the KLE coefficients are conditioned in order to guarantee the correlation structure in the entire domain. The parallelization of this technique is developed and the advantages are highlighted. Through this, the computational complexity is drastically reduced. The mean-square global block error is computed. It provides accurate results when using a family of covariance functions with compact support. Some numerical examples to study the performance of the proposed approach are included.