CCA-MTFCN: A Robotic Pushing-Grasping Collaborative Method Based on Deep Reinforcement Learning

Robotic grasping in dense clutter is often infeasible because of the occlusion and stacking of objects. Directly grasping the stacked objects may cause collisions and result in low efficiency and high failure rates. In practice, the lateral robotic push can separate stacked objects to create collision-free grasp affordances. Inspired by this, we devise a method called CCA-MTFCN based on deep reinforcement learning, which can learn the synergies between pushing and grasping policies to complete the task of removing all objects in a heavily cluttered environment. Specifically, a hard parameter-sharing Multi-Task Fully Convolutional Network (MTFCN) is proposed to model the action-value function, then multi-scale feature fusion mechanism is implemented in it which can enhance visual perception capability in cluttered environments. Moreover, a new reward function based on connected component analysis (CCA) is designed to effectively evaluate the quality of push actions in pushing-and-grasping collaboration. This enables us to explicitly encourage pushing actions that aid grasping thus improving the efficiency of sequential decision-making. Our approach was trained in simulation through trial-and-error, and evaluation experiments for object removal tasks in dense clutter demonstrate that our proposed method outperforms several baseline approaches in terms of task completion rate, grasp success rate, and action efficiency, which also has the capability to generalize to new scenarios.