Adaptation II: High-Dimensions and Deep Neural Networks

This chapter further develops the idea of constructing flexible prior distributions. We consider the question of finding posterior contraction rates that adapt to certain unknown structures present in the data. Two main situations are investigated: sparsity and compositional structures. For adaptation to sparsity in high-dimensional regression models, we discuss spike-and-slab type priors and (briefly) continuous shrinkage priors. For adaptation in the setting of compositions of functions, we consider a number of deep learning priors, including deep neural networks with ReLU activation and deep Gaussian process priors.