Convergence to Equilibrium

This chapter is concerned with the large time behavior of Markov chains, including the computation of their limiting and stationary distributions. Here the notions of recurrence, transience, and classification of states introduced in the previous chapter play a major role. We also derive quantitative bounds for the convergence of a Markov chain to its stationary distribution. The Markov Chain Monte Carlo (MCMC) method presented in Sect. 6.2 is widely used for statistical estimation based on the Markov property.