To address the challenge of balancing “exploitation” and “exploration” in reinforcement learning-based maneuver decision-making methods, this paper proposes a Dynamical Receding Horizon Control - Proximal Policy Optimization (DRHC-PPO) algorithm. By interacting with the Beyond-Visual-Range (BVR) air combat environment, the algorithm explores excellent maneuver intention solutions, which guide the maneuver decision-making. First, the PPO algorithm interacts with the BVR air combat environment and a reward function is utilized for evaluation and training to derive the intention solution. Subsequently, the intention decision result is matched with an intention model to guide underlying maneuver decision-making. Finally, the dynamical receding horizon control method traverses the discretized control variables of the Unmanned Combat Aerial Vehicle (UCAV) model to compute a near-optimal feasible maneuver solution. Simulation results demonstrate that the proposed method achieves effective convergence and enables occupancy maneuver decision-making.