This chapter introduces another major category of reinforcement learning algorithms: policy-based RL. First, we will try to smoothly transition from value-based RL by discussing issues with value-based RL and how such issues can be addressed with policy-based RL. Next, we will get familiar with the basic concepts in policy-based RL and reveal the major steps of policy-based RL studies: objective function construction, policy definition, training via policy improvements, and algorithm improvements. With the understanding of these steps, we will first introduce an objective function to derive the policy gradient. Based on it, the deduction of the policy gradient theorem will be presented. Then, we will show the Monte Carlo implementation with the derived policy gradient theorem. Next, we will exhibit issues with the simple Monte Carlo implementation, which can be improved in two different directions: policy and value. As for policy, we will introduce more forms of objective functions and show a more widely accepted policy gradient theorem. For value, we will discuss more policy evaluation methods for reducing the variance. Various classic policy gradient algorithms such as REINFORCE, REINFORCE with baseline, Actor-Critic (TD(0) and TD( \(\lambda \) )) will be introduced.