As robotic technologies advance, robotic arm manipulation tasks in complex environments become increasingly important. This paper presents a new collaborative pushing and grasping strategy to address two major challenges robotic arms encounter in complex environments: invisibility and tight enclosure of target objects. We propose a generator-evaluator network architecture based on multiple action primitives, incorporating hierarchical reinforcement learning and Double DQN techniques to optimize the operational efficiency of the robotic arm. By introducing a new action primitive to prioritize pushing operations around the topmost objects in space, our approach effectively reveals and locates obscured target objects. Additionally, utilizing the Spatial Correlation Test (SCT), our method explores all possible collision-free operational positions in the action space, selecting the optimal position from among them. Experimental results demonstrate that our approach achieves high efficiency and success rates in scenarios with randomly distributed objects, objects tightly enclosed by others, and invisible objects. This strategy enhances the speed and accuracy of tasks and improves the robot’s adaptability to complex environments.