Multi-agent Reinforcement Learning Algorithm Based on Role Parameter Sharing

In multi-agent reinforcement learning, scalability is a significant challenge. Applying reinforcement learning algorithms in large-scale multi-agent systems requires consideration of algorithm efficiency. This paper attempts to reduce network parameter quantity and training time from the perspective of parameter sharing. However, parameter sharing may also bring some disadvantages, such as policy similarity, which may limit the formation of cooperative policies and affect the efficiency of the entire system. To address the problems caused by parameter sharing, this paper proposes a multi-agent reinforcement learning algorithm with role parameter sharing (R-PS). By dynamically dividing roles and selecting objects for parameter sharing, R-PS automatically identifies intelligent agents that may generate beneficial parameter sharing and conducts partial parameter sharing within the roles. Meanwhile, intrinsic rewards are introduced to increase the diversity of agent strategies. Experimental results show that R-PS can effectively divide roles, benefit from partial parameter sharing, and form diverse policies through intrinsic rewards. It also reduces training time, produces excellent performance and collaboration.