Policy Gradient Algorithms

Up to now, the book has focused on model-based and model-free methods. All the algorithms using these methods estimate the state or state-action values for a given current policy as the first step. In the second step, these estimated values are used to find a better policy by choosing the best action in a given state. These two steps are carried out in a loop until no further improvement in values is observed. In this chapter, you look at a different approach for learning optimal policies, by directly operating in the policy space. You will learn to improve the policies without explicitly learning or using state or state-action values.