Robotic Grasping Decision Making Assisted by AI and Simulation

Artificial intelligence (AI) and simulation technologies are currently experiencing a rise within robotic manipulation. This work introduces the autonomous grasping point detection (GPD) system proposed in the HARTU project, which is assisted by simulation and AI. We propose a two stage pipeline: In the first one grasping points are extracted with deep learning (DL) techniques and analytical sampling methods in isolated objects, and tested in simulation. Then, we develop a deep reinforcement learning (DRL) decision module, trained in simulation, to select the best grasping points in complex scenes. The whole pipeline is integrated within the ROS2 ecosystem.