Learning and Fast Adaptation for Air Combat Decision with Improved Deep Meta-reinforcement Learning

Artificial intelligence plays a pivotal role in autonomous decision-making and control within visual range in air combat. Conventional maneuver decision methods that require an exact dynamic model are limited to the point-mass model aircraft. Model-free reinforcement learning enables air combat research to address practical situations featuring high-fidelity nonlinear flight dynamic models. This study addresses the problem of how an agent can maneuver and control in unseen tasks, which is crucial for improving the adaptability of the agent to complex situations that may not be encountered during training. Accordingly, a meta-reinforcement learning framework is first introduced in air combat decisions for training the agent to control the offensive situation and knock down bandits. In particular, the meta-training tasks are constructed based on typical engagement situations in order to learn basic fighter maneuver in air combat. The corresponding reward function for each of these tasks is shaped based on the potential function of situation assessment. In contrast, the meta-evaluation tasks are designed with uniform but harder goal conditions, which require the integration of meta-training. In implementation, a meta-reinforcement learning algorithm is proposed in order to facilitate fast adaptation to multiple tasks. In the context of meta-evaluation tasks, the success rate of our algorithm is approximately 80%. Furthermore, comparative results demonstrate the effectiveness of our method. An ablated study additionally identifies the efficiency of the meta-reinforcement learning framework. Visualization and analysis of the air combat process are presented. Finally, it can be concluded that the unseen tasks are performed with the combinatorial skills learned in meta-training in the context of air combat issues.