ε-Maximum Critic Deep Deterministic Policy Gradient for Multi-agent Reinforcement Learning

In Multi-Agent Reinforcement Learning, the agents are vulnerable to the other agents and the training environment, which can lead to agents’ policy achieving a local optima easily and poor convergence efficiency. To tackle the above challenges, we propose a novel algorithm, g-Maximum Critic Multi-Agent Deep Deterministic Policy Gradient algorithm (g-M2DDPG), which leverages a new critic technique called g-Maximum Critic to balance the exploitation and exploration in updating Q-value function. We empirically evaluate our algorithms in three kinds of mixed cooperative and communication environments. These experimental results demonstrate that our algorithms significantly accelerates the learning process and outperform existing baseline algorithm MADDPG.