Scholarlink

<p>Deep Reinforcement Learning (DRL) methods are increasing the attention of the research community to control complex multivariable nonlinear systems with interactions. A primary concern regarding DRL-based methods is how well they can generalize their experiences to perform well on never-seen-before tasks (i.e., zero-shot generalization) in the same environment. Previous research has empirically demonstrated that a Deep Deterministic Policy Gradient (DDPG) controller design addresses this issue efficiently for a benchmark Quadruple-Tank Process. Nevertheless, compared to the Practical Nonlinear Model Predictive Control (PNMPC), the DDPG controller exhibits a slower tracking performance by a significant amount of time. This research study will investigate a more detailed comparison between both controllers to emphasize the differences and find improvements. The conducted sensitivity analysis shows that the DDPG controller exhibits robustness to the model parameter uncertainty, in contrast with the PNMPC which relies on the plant model. Furthermore, an analysis of the change in the control inputs reveals that it is possible to design a new state and reward function for the DDPG controller. The comparison with this new design shows that the DDPG can achieve comparable tracking performance to the PNMPC, without making any additional changes to the original methodology.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

DDPG and PNMPC controller design comparison for a Quadruple-tank process control benchmark

  • 期刊论文

  • 发表日期：

    2025年6月24日

查看全文
  • Javier Machacuay,
  • William Ipanaqué

摘要

Deep Reinforcement Learning (DRL) methods are increasing the attention of the research community to control complex multivariable nonlinear systems with interactions. A primary concern regarding DRL-based methods is how well they can generalize their experiences to perform well on never-seen-before tasks (i.e., zero-shot generalization) in the same environment. Previous research has empirically demonstrated that a Deep Deterministic Policy Gradient (DDPG) controller design addresses this issue efficiently for a benchmark Quadruple-Tank Process. Nevertheless, compared to the Practical Nonlinear Model Predictive Control (PNMPC), the DDPG controller exhibits a slower tracking performance by a significant amount of time. This research study will investigate a more detailed comparison between both controllers to emphasize the differences and find improvements. The conducted sensitivity analysis shows that the DDPG controller exhibits robustness to the model parameter uncertainty, in contrast with the PNMPC which relies on the plant model. Furthermore, an analysis of the change in the control inputs reveals that it is possible to design a new state and reward function for the DDPG controller. The comparison with this new design shows that the DDPG can achieve comparable tracking performance to the PNMPC, without making any additional changes to the original methodology.