<p>This paper studies the problem of action noise in model-based offline reinforcement learning, i.e., the actions recorded in the transition trajectories are polluted with noise. Though this is a relatively new problem in the literature, it has become a relevant issue since offline reinforcement learning has become more and more widely used. This is particularly important in applications where an offline dataset collected without the intention to improve the policy is repurposed for reinforcement learning. This paper presents an error analysis for the value function due to the action noise and provides numerical studies. This work also prepares for further developments of novel algorithms for addressing action noise.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An Analysis of Offline Model-Based Learning with Action Noise

  • Haoya Li,
  • Tanmay Gangwani,
  • Lexing Ying

摘要

This paper studies the problem of action noise in model-based offline reinforcement learning, i.e., the actions recorded in the transition trajectories are polluted with noise. Though this is a relatively new problem in the literature, it has become a relevant issue since offline reinforcement learning has become more and more widely used. This is particularly important in applications where an offline dataset collected without the intention to improve the policy is repurposed for reinforcement learning. This paper presents an error analysis for the value function due to the action noise and provides numerical studies. This work also prepares for further developments of novel algorithms for addressing action noise.