The Foundation: Markov Decision Processes

As discussed in Chapter 1 , reinforcement learning involves sequential decision-making. This chapter formalizes the notion of using stochastic processes under the branch of probability that models sequential decision-making behavior. Although most of the problems you'll study in reinforcement learning are modeled as Markov decision processes (MDP), this chapter starts by introducing Markov chains (MC) followed by Markov reward processes (MRP). Next, the chapter discusses MDP in-depth while covering model setup and the assumptions behind MDP.