Bayesian Hierarchical Modelling

In the previous chapters, our statistical procedure was very simple: define a prior probability distribution for the parameters \(p[\theta ]\) and a likelihood function \(L[\theta ]=p[y|\theta ]\) , and that was it. Bayes’ Theorem then told us what the posterior distribution would be once we received the data: \(p[\theta |y] \propto p[\theta ] L[\theta ]\) . The prior for the parameter vector was always a fully specified distribution, e.g. the product of known univariate Gaussians. In Bayesian hierarchical modelling (BHM), we do not specify the prior that directly. Instead we make the prior distribution depend on other parameters, which we call hyperparameters. Here is a table of the differences: \(\displaystyle \begin {aligned}{} & \underline {\text{Non-hierarchical}} \quad && \underline {\text{Hierarchical}} \\ \text{Likelihood} \quad & p[y|\theta ] && p[y|\theta ] \\ \text{Prior} \quad & p[\theta ] && p[\theta |\xi ] \\ \text{Hyperprior} \quad & - && p[\xi ] \end {aligned} \)