Infinite-dimensional distances and divergences between positive definite operators, Gaussian measures, and Gaussian processes

This paper presents a survey of recent results on the generalization of distances and divergences on the set of symmetric, positive definite (SPD) matrices to the infinite-dimensional setting of positive definite Hilbert–Schmidt operators on a Hilbert space. Our focus here is on the affine-invariant Riemannian metric and the Log–Determinant divergences. Key components in the proper formulation of the infinite-dimensional distances and divergences include the concepts of extended Hilbert–Schmidt and trace class operators, extended Hilbert–Schmidt inner product and norm, and extended Fredholm and Hilbert–Carleman determinants. On the set of positive trace class operators, the resulting affine-invariant Riemannian distance and Alpha Log–Det divergences can be viewed as regularized versions of the exact Fisher–Rao distance and Rényi divergences, respectively, between equivalent centered Gaussian measures on a Hilbert space. In the case of Gaussian measures corresponding to Gaussian processes with squared integrable paths, the regularized infinite-dimensional distances and divergences can be consistently estimated from finite-dimensional versions, with dimension-independent sample complexities, via the methodology of reproducing kernel Hilbert spaces (RKHS). We also discuss the practical applications of this framework in machine learning and computer vision in the setting of RKHS covariance operators.