The Expectation Maximization Algorithm

In many practical applications, not all the variables needed for maximum likelihood (ML) estimation are available but observations y provide data for estimation using a known statistical model. In such cases, the expectation maximization algorithm allows us to use an initial guess of the parameter values to recursively improve our estimate of the parameters of a known distribution in the absence of a complete set of data. The algorithm can also be used in some cases where it is difficult to solve for the ML estimate. This chapter provides an introduction to the EM algorithm that includes how the algorithm is simplified for the exponential family of distributions. It also includes the use of the EM algorithm to fit data with a mixture of distributions.