Scholarlink

<p>Current constrained reinforcement learning (RL) algorithms face challenges such as slow policy learning, unstable training, and heavy hyperparameter tuning due to their nonconvex optimization nature, often resulting in suboptimal convergence or divergence. To address these issues, this paper proposes the feasible optimization with monotonic improvement (FOMI) algorithm, which guarantees constraint satisfaction and monotonic policy improvement. First, the primal problem is simplified using a Taylor expansion and reconstructed with a trust region constraint, reducing complexity and improving optimization characteristics. Then, a feasible optimization framework is established for the reconstructed problem, which is decomposed into performance improvement and feasibility recovery subproblems to obtain a policy that improves performance while satisfying constraints. An analytic solution for the reconstructed problem is derived, eliminating backpropagation during network training and accelerating policy learning. Building upon this framework, FOMI incorporates neural networks as function carriers for continuous control tasks. Simulations validate that FOMI exhibits excellent training stability and a two-fold increase in learning speed compared to baselines. Real-world experiments verify the effectiveness of FOMI, highlighting its potential for tackling complex real-world tasks.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Feasible Reinforcement Learning for Safety Ensurance with Stabilized Training

  • 期刊论文

  • 发表日期：

    2026年1月10日

查看全文
  • Zuoshuai Wang,
  • Yao Lyu,
  • Congsheng Zhang,
  • Zhe Xin,
  • Jiaxin Gao,
  • Beiyan Jiang,
  • Shengbo Eben Li

摘要

Current constrained reinforcement learning (RL) algorithms face challenges such as slow policy learning, unstable training, and heavy hyperparameter tuning due to their nonconvex optimization nature, often resulting in suboptimal convergence or divergence. To address these issues, this paper proposes the feasible optimization with monotonic improvement (FOMI) algorithm, which guarantees constraint satisfaction and monotonic policy improvement. First, the primal problem is simplified using a Taylor expansion and reconstructed with a trust region constraint, reducing complexity and improving optimization characteristics. Then, a feasible optimization framework is established for the reconstructed problem, which is decomposed into performance improvement and feasibility recovery subproblems to obtain a policy that improves performance while satisfying constraints. An analytic solution for the reconstructed problem is derived, eliminating backpropagation during network training and accelerating policy learning. Building upon this framework, FOMI incorporates neural networks as function carriers for continuous control tasks. Simulations validate that FOMI exhibits excellent training stability and a two-fold increase in learning speed compared to baselines. Real-world experiments verify the effectiveness of FOMI, highlighting its potential for tackling complex real-world tasks.