Multi-Agent Continual Deep Reinforcement Learning with Artificial Potential Fields and Switching Control for a Cooperative Robotic Task
摘要
Autonomy and intelligence are the key directions of modern manufacturing, with Deep Reinforcement Learning (DRL) providing important technical support for robots to acquire skills autonomously. Multi-agent reinforcement learning (MARL) offers a principled framework for coordinated decision-making in collaborative robotics, yet it often suffers from poor sample efficiency and brittleness when demonstrations are limited. To address these issues, we propose a unified framework that couples Multi-Agent Artificial Potential Fields (MAPF) with Behavior Cloning (BC), continual adaptation, and a safety-oriented switching controller. Specifically, MAPF generates structured expert rollouts for joint-space bimanual control under explicit joint constraints, yielding an effectively unlimited demonstration stream. We then train a policy via a critic-gated BC loss and integrate an online continual refreshing mechanism to mitigate BC overfitting during long-horizon training. Finally, we deploy a switching controller that uses the learned policy as the default solver and falls back to MAPF when the time budget is exceeded, improving reliability while retaining low runtime. We evaluate the approach on two RoboSuite bimanual tasks to test generalization under goal variations. Under identical observations and rewards, we additionally include a strong SAC-based CTDE/off-policy backbone within the same MABCRL pipeline (MA(SAC)BCRL) for comprehensive comparison. Across both tasks, the proposed MABCRL training improves learning stability and final performance compared with plain MARL and naive BC. On ENV1, MA(DDPG)BCRL achieves