Hidden Markov Models

A Markov model describes a sequence of possible states with random transition between states where the probability of the current state depends only on the preceding state and is independent of the way the preceding state was reached. The state space can be continuous or discrete, but we restrict our analysis to the case of a discrete state space. A state-space grid with \(N\) grid points can be used to obtain an approximate representation of a continuous state space. This representation has a one-to-one correspondence between the discrete state space and the set of integers \(\left\{1, 2, \dots ,N\right\}\) . If the states of the system can only be observed indirectly through measurements that randomly depend on the state, we have a hidden Markov model (HMM). The HMM can be used to estimate the state of a dynamic system when no state-space model of the system is available provided that the probabilities of transition between states and measurements are known.