2v2 Close Air Combat Decision-Making Based on Improved MAPPO Algorithm

Aircraft cluster air warfare is a complex and challenging combat scenario. Reinforcement learning is applied to unmanned cluster control because of its powerful dynamic decision-making and control capabilities. However, for the scenario described above, the multi-agent reinforcement learning algorithm still has issues such as local optima and long training times. To address the above issues, our work improves the Multi-Agent Proximal Policy Optimization(MAPPO) algorithm. Specifically we apply a mechanism to reduce the dimensionality of actions and corresponding values, and design an adaptive reward function which can help the agent maintain a good balance between attack and defense. In addition we built a 2V2 simulation scenario of close air combat to evaluate our algorithm. The experimental results demonstrate that the models trained by our algorithm are more effective in decision-making performance.