Affordance Maps for Picking or Separating Entangled Objects

In this paper, we propose an autonomous, effective and general approach for picking up tangled-prone objects for industrial bin picking. First, we learn PickNetFlexiblyPickNet - a network that maps the visual observation to pixel-wise possibilities of picking isolated objects or separating tangled objects and infers the corresponding grasp. Then, we propose two effective separation strategies: Dropping the entangledEntangled objects into a buffer bin to reduce the degree of entanglement; Pulling to separate the entangledEntangled objects in the buffer bin plannedPullNetspatial action by PullNet - a network that predicts position and direction for pulling from visual input. To efficiently collect data for training PickNet and PullNet, we embrace the self-supervised learning paradigm using an algorithmic supervisorDataphysics simulator in a physics simulator. Real-worldReal-worldbaselines experiments show that our policy can dexterously pick up tangled-prone objects with success rates of 90%. We further demonstrate the generalizationGeneralizationfailure of our policy by picking a set of unseen objects. Supplementary material, code, and videos can be found at https://xinyiz0931.github.io/tangle .