Scholarlink

<p>A novel Reinforcement Learning (RL) approach Multi-agent Joint Control with State-Action Embedding (MASAE) is proposed in this paper to address the sample-efficiency issue of RL in robot control. It combines the relative entropy regularization and high update-to-data (UTD) ratios in one multi-agent framework to accelerate the learning process while naturally mitigating the overestimation of value functions caused by high UTD ratios by multiple agents. The state-action embeddings are employed to adaptively abstract the hidden features behind the state-action space for enhanced learning efficiency. Evaluated by several simulated benchmark control tasks and a real-world Unitree Go1 quadruped robot system, MASAE demonstrates significant advantages in learning capability and sampling efficiency compared to various related RL baselines, indicating its potential in learning challenging real-world robot systems with a limited number of samples. The open-source code of MASAE is available at <a href="https://github.com/AdrienLin1/MASAE">https://github.com/AdrienLin1/MASAE</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Sample-efficient multi-agent reinforcement learning with high update-to-data ratio and state-action embedding

  • 期刊论文

  • 发表日期：

    2025年12月9日

查看全文
  • Chenyang Miao,
  • Yingzhuo Jiang,
  • Yunduan Cui,
  • Yidong Chen,
  • Tianfu Sun

摘要

A novel Reinforcement Learning (RL) approach Multi-agent Joint Control with State-Action Embedding (MASAE) is proposed in this paper to address the sample-efficiency issue of RL in robot control. It combines the relative entropy regularization and high update-to-data (UTD) ratios in one multi-agent framework to accelerate the learning process while naturally mitigating the overestimation of value functions caused by high UTD ratios by multiple agents. The state-action embeddings are employed to adaptively abstract the hidden features behind the state-action space for enhanced learning efficiency. Evaluated by several simulated benchmark control tasks and a real-world Unitree Go1 quadruped robot system, MASAE demonstrates significant advantages in learning capability and sampling efficiency compared to various related RL baselines, indicating its potential in learning challenging real-world robot systems with a limited number of samples. The open-source code of MASAE is available at https://github.com/AdrienLin1/MASAE.