Variational Approximations

This chapter investigates a popular approach for computing an approximation of the posterior distribution, namely variational Bayes. Given a class of sufficiently simple distributions, the variational posterior is a best approximation of the posterior within this class, in the Kullback-Leibler sense. Following recent work in the area, we provide generic tools to derive contraction rates for the variational posterior, tools that can be applied for instance to nonparametric models and mean-field variational classes. We then also discuss the setting of high-dimensional linear regression with spike-and-slab priors and spike-and-slab variational class.