<p>The rapid growth of air transportation has gradually become one of the bottlenecks in the air traffic management system. In trajectory-based operations, the allocation of airspace constraints for civil aircraft has a direct impact on the effectiveness of airspace operations in the pre-flight phase. However, the current research is still facing many challenges when allocating airspace resources, taking into account the flight preferences and operational objectives of various stakeholders. On one hand, it is difficult for the current research to maximize the overall operational efficiency of airspace, and on the other hand, the efficiency of the deep reinforcement learning algorithm for allocating airspace constraints still requires improvement. Therefore, this paper proposes a multi-agent reinforcement learning algorithm to improve the operational efficiency of airspace by training agents to learn flight strategies. Firstly, the reward function is designed to decrease flight conflicts and flight delays and alleviate the imbalance between airspace demand and capacity, with the aim of improving the operational efficiency of various stakeholders. Secondly, the multi-head attention and the self-attention mechanisms are introduced during the training process. These mechanisms enable the agent to flexibly focus on different state information of the agent to improve the performance of the multi-agent reinforcement learning algorithm. Finally, the algorithm is compared and tested in a simulated airspace environment. Experimental results show that compared to the traditional DMARL and A-DMARL algorithms, the proposed algorithm has higher learning efficiency and convergence speed, and the proposed algorithm is able to effectively decrease flight conflicts, reduce flight delays, and alleviate the imbalance between airspace demand and capacity.</p>