An Efficient MADDPG with Episode-Parallel Interaction and Dual Priority Experience Replay

Multi-agent Deep Deterministic Policy Gradient (MADDPG) is a common multi-agent deep reinforcement learning algorithm applied in both cooperative and competitive scenarios. However, the frequent interactions with the environment and indiscriminate sampling for training models will lead to poor training efficiency and low convergence performance. To overcome above limitations, this paper proposes an efficient MADDPG with episode-parallel interaction and dual priority experience replay (EIDPER-MADDPG), which can achieve a better convergence performance in a shorter training time. Firstly, we devise a parallel interaction architecture to utilize multiple processes for collecting experiences and learning from them repeatedly in one sampling. Secondly, considering the contributions of samples from two perspectives in model training and task scenarios, we redesign a dual priority experience replay for evaluating samples’ importance, which provides more valuable samples for training and enhances the convergence performance. Furthermore, we conduct simulations to demonstrate the effectiveness of the proposed algorithm in terms of training efficiency and convergence performance.