In the domain of multi-agent reinforcement learning, the scalability of multi-agent systems presents challenges for conventional policy-based methods. As the scale increases, these methods struggle due to the growing state space and partially observable Markov decision process, which are further exacerbated by the interference between observations. This paper introduces a novel framework for enhancing multi-agent proximal policy optimization with a hard attention network. All of the features in the observation vector of one particular agent can be re-sorted according to their calculated attention values, and only those are relatively important are preserved and aggregated for decision making. Within the resorting and pruning manipulations based on hard attention, the input space of actor network is efficiently reduced, leading to faster and more stable learning for policy and critics. Our framework outperforms the vanilla multi-agent proximal policy optimization algorithm on cluster confrontation tasks of various scales and ensures training success even under extreme observation interference.