Fusion of Transformer in SAC Reinforcement Learning for Online Game Confrontation Algorithm
摘要
When facing new interception threats, traditional rule-based or analytical methods are insufficient in terms of complexity and adaptability in confrontation. This paper proposes an intelligent game confrontation algorithm that integrates the Transformer model with the Soft Actor-Critic (SAC) reinforcement learning method for a one-to-three head-on interception scenario involving high-speed vehicles. The algorithm uses the multi-head attention mechanism of the Transformer to extract threat information from the blue targets and effectively handles the state space dimensionality issues caused by changes in the number of blue targets using a masking mechanism. In the design of the action space and reward function, the algorithm uses angle of attack, velocity tilt angle, and discrete maneuver switches as output commands, and designs a multi-dimensional reward function based on energy management, action timing optimization, and threat assessment. Experimental results show that the algorithm outperforms traditional procedural maneuver algorithms in terms of confrontation success rate and energy consumption, demonstrating good confrontation effectiveness and adaptability. However, the model’s interpretability and scenario adaptability still need further optimization to enhance its generalization performance.