Since its introduction in a seminal 1977 paper by Dempster, Laird, and Rubin, the Expectation- Maximization (EM) algorithm has been widely utilized as an iterative scheme to obtain maximum-likelihood estimates for parametric models in which a portion of the data is either unobserved or latent. We briefly review ML estimation for i.i.d. normal data, then present the algorithm proper through the lens of a finite Gaussian mixture model. Following a proof of the algorithm’s monotonicity in the updates to the likelihood, we discuss briefly other scenarios in which the algorithm is useful, as well as proposed extensions of the algorithm from the literature. We conclude with an application to the classification problem of data arising from a finite mixture of multivariate Gaussian distributions with equal covariance matrices.
This talk will be accessible to Juniors and Seniors.