Reinforcement learning (RL) agents often struggle to learn effective pursuit strategies in complex environments due to partial observability and coordination challenges. To address this, we propose PER-DMADDPG (Prioritized Experience Replay Distributed Multi-Agent Deep Deterministic Policy Gradient), a distributed framework with partitioned policy learning based on the initial visibility of the evader relative to each pursuer’s field of view (FOV). We design three specialized sub-policies for different observability conditions: (1) partially visible, (2) fully visible, and (3) fully occluded. Prior to each episode, agents autonomously select the most appropriate policy based on local observations, enabling decentralized coordination. To enhance learning, we introduce a hierarchical experience replay mechanism composed of both general and sub-policy-specific buffers. We also employ curriculum learning, progressively increasing task complexity to facilitate robust policy acquisition. Experimental results demonstrate that PER-DMADDPG outperforms existing methods in pursuit success rate and convergence speed. Real-world experiments with four non-holonomic UGVs further validate its effectiveness under physical constraints.