Parameter sharing in multi-agent reinforcement learning (MARL) holds the promise of achieving scalable multi-agent cooperation but also leads to the homogenization of agents’ behaviors, which severely hinders the model’s ability to learn cooperative strategies. To alleviate this, previous works have explored the concept of roles, enabling agents to execute distinct role-specific policies. However, there are still limitations in the role assignment mechanisms and model complexity. In this paper, we introduce Mixture of Experts (MoE) into multi-agent systems and propose a plug-and-play framework called Multi-Agent MoE. It innovatively allows the agent to select multiple experts with different skills concurrently, significantly expanding the space of available policies and boosting the agent’s capacity to handle complex situations. The lightweight architecture further contributes to its high computational efficiency. These advances enable our proposed method to significantly outperform existing state-of-the-art approaches on multiple benchmarks.