Scholarlink

<p>We address the crucial yet underexplored stability properties of the Hamilton–Jacobi–Bellman (HJB) equation in model-free reinforcement learning contexts with a focus on Lipschitz continuous optimal control problems. By an established connection between Lipschitz continuous optimal control problems and classical optimal control problems, we provide new insights into the stability of the value functions. Specifically, we investigate the generic convergence property as well as the rate of convergence of value functions as the Lipschitz constraint increases. Moreover, we present a generalized Hamilton–Jacobi based reinforcement learning framework that accommodates general <InlineEquation ID="IEq1"><EquationSource Format="MATHML"><math><msup><mi>L</mi><mi>p</mi></msup></math></EquationSource><EquationSource Format="TEX">$L^{p}$</EquationSource></InlineEquation> geometries in the control regularization. To evaluate the stability properties and performance of our proposed method, we conduct various numerical experiments using benchmark examples and compare its efficacy with existing approaches.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

On the stability of Lipschitz continuous control problems and its applications to reinforcement learning

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月17日

查看全文
  • Namkyeong Cho,
  • Yeoneung Kim

摘要

We address the crucial yet underexplored stability properties of the Hamilton–Jacobi–Bellman (HJB) equation in model-free reinforcement learning contexts with a focus on Lipschitz continuous optimal control problems. By an established connection between Lipschitz continuous optimal control problems and classical optimal control problems, we provide new insights into the stability of the value functions. Specifically, we investigate the generic convergence property as well as the rate of convergence of value functions as the Lipschitz constraint increases. Moreover, we present a generalized Hamilton–Jacobi based reinforcement learning framework that accommodates general Lp$L^{p}$ geometries in the control regularization. To evaluate the stability properties and performance of our proposed method, we conduct various numerical experiments using benchmark examples and compare its efficacy with existing approaches.