More on Markov Chain Monte Carlo

In estimating parameters of a process from empirical data Maximum Likelihood Estimation (MLE) is often used. But it may be desirable to temper MLE with, for example, historical data. Enter Bayesian Inference (BI) based on Bayes’ Theorem. The concepts of prior probability, posterior probability, Maximum A posteriori Estimator, and conjugate distribution are explained and illustrated through two applications. The open software package Pymc3 is demonstrated which makes BI available to non-professionals in MCMC. Gibbs Sampling is a technique for sampling from a joint distribution when this may be difficult to do by direct means but its conditional densities are available. The method samples from each component in succession, one at a time, always using the most recent sampled values. Monte Carlo integral estimation is effective for multi-dimensional integrals. The integral can be regarded as the expectation of its integrand with respect to an arbitrary sampling density. The error of the method is on the order of \(n^{-1/2}\) regardless of integrand or dimensionality. The method is illustrated with several examples. Two variance reduction techniques are covered: control variate and importance sampling. We also show how to use MCMC to obtain candidates from the sampling density when direct methods are not available.