<p>The 5th Generation (5G) Mobile Network, coupled with the Internet of Things (IoT), is a heterogeneous environment prone to various security vulnerabilities and frequent attacks. Therefore, analyzing attackers’ intrusion intentions and penetration behaviors is crucial for guiding network security defenses. However, existing penetration path constructions mostly rely on a single agent or consider a single type of network, which cannot comprehensively assess the impact of group attack behaviors and system vulnerability combinations on security. To address this issue, a multi-agent reinforcement learning approach is presented for collaborative path penetration. This solution leverages network situational information to guide policy learning, improving the overall path penetration quality. Agents adopt a hierarchical structure of primary and subordinate roles. The primary agent can observe the entire environmental state and formulate top-level collaborative strategies for the group, while subordinate agents learn individual policies based on assigned tasks and team rewards and interact with the environment to learn. Through this mode, agents coordinate with each other to select and continuously improve their strategies in penetration. Experimental results show that compared with single-path penetration under a single agent, the proposed method generates multiple penetration paths with higher attack efficiency, decision stability, and task completion rate.</p>