A Scalable Multi-agent Reinforcement Learning Approach Based on Value Function Decomposition

Due to limitations in input dimensionality of fully connected neural networks and joint action value estimation, most existing multi-agent deep reinforcement learning methods based on value function decomposition assume that the number of agents in the environment remains constant during training. However, in reality, the number of agents may vary due to agent failure or reinforcement. Therefore, the unshaped scenario where the number of agents varies is more general. In this paper, we propose a multi-agent deep reinforcement learning method based on value function decomposition that is suitable for unshaped scenarios. We introduce an attention mechanism to process observation data with varying dimensions in the agent network, and incorporate historical information to compute attention query vectors. We propose a shared-parameter attention-based self-weighted network to compute joint action values. The proposed method achieves better results than baseline methods in multiple simulation scenarios with varying numbers of agents.