Data-Driven Reinforcement Learning for Mission Engineering and Combat Simulation

Historically, operational analysis has largely utilized human operators and subject matter experts (SMEs) with the military knowledge and expertise necessary to make realistic decisions about combat engagements or missions. Recent operational challenges have prompted a renewed interest in machine-assisted mission engineering tools that can advance decision-making and innovation in defense. These tools have the ability to simulate high-fidelity combat engagements to analyze operational concepts, platforms, systems, and capabilities in multiple domains to develop optimal solutions to the intricate and difficult issues within the field of mission engineering. The ability of reinforcement learning (RL) to master highly complex win-loss games has highlighted its potential within the field of Mission Engineering. Agents trained through RL can interpret the environment and learn to navigate the game through experience, a success that, in theory, should translate well to combat simulations. However, in practice, using combat simulation tools in combination with RL methods poses particular challenges due to the difficulty of agent interaction with simulators not built for RL engagement. However, offline—or data-driven—RL overcomes these challenges by removing the agent’s dependence on interaction with the environment, and instead using historical data for training. This study proposes training an agent with synthetic combat simulation data by using offline, or data-driven, RL.