The policy gradient method is another commonly used method in deep reinforcement learning. Unlike value-based methods, it directly updates the policy by calculating the gradient of the expected total reward with respect to the policy parameters, aiming to solve some shortcomings of value-based methods. This chapter is to introduce several policy gradient methods. Firstly, the definition of policy gradient is given. Secondly, the basic form of policy gradient is deduced. Finally, three types of policy gradient methods are presented, including actor-critic-based policy gradient methods, deep deterministic policy gradient methods, and asynchronous reinforcement learning methods. In actor-critic-based policy gradient methods, the actor updates actions based on the policy gradient algorithm, and the critic evaluates actions based on the value function method. In deep deterministic policy gradient, deterministic policy gradient (DPG) algorithm and deep deterministic policy gradient (DDPG) algorithm are introduced. In asynchronous reinforcement learning methods, asynchronous advantage actor-critic (A3C) is analyzed.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Policy Gradient Algorithms

  • Jianhua Li

摘要

The policy gradient method is another commonly used method in deep reinforcement learning. Unlike value-based methods, it directly updates the policy by calculating the gradient of the expected total reward with respect to the policy parameters, aiming to solve some shortcomings of value-based methods. This chapter is to introduce several policy gradient methods. Firstly, the definition of policy gradient is given. Secondly, the basic form of policy gradient is deduced. Finally, three types of policy gradient methods are presented, including actor-critic-based policy gradient methods, deep deterministic policy gradient methods, and asynchronous reinforcement learning methods. In actor-critic-based policy gradient methods, the actor updates actions based on the policy gradient algorithm, and the critic evaluates actions based on the value function method. In deep deterministic policy gradient, deterministic policy gradient (DPG) algorithm and deep deterministic policy gradient (DDPG) algorithm are introduced. In asynchronous reinforcement learning methods, asynchronous advantage actor-critic (A3C) is analyzed.