When Vision Mamba Meets Robotic Grasping: An Efficient Grasp Detection Network for Real-Time Robotic Manipulation
摘要
The robot grasping paradigm faces a trade-off between limited local awareness, i.e., the convolution al neural network (CNN) based grasping approach, and high computational costs, i.e., transformer-based grasping method. In this paper, we propose a novel grasp detection network based on Vision Mamba (VM), namely VM-Grasp, to effectively integrate local and global information, thereby enhancing the accuracy and efficiency of grasp detection. We leverage two key designs of VM, the input image is divided into multiple patches and fed into the model as a sequence, effectively compressing the visual representation. Subsequently, we employ bidirectional state space modeling to effectively identify global context patterns and key grasping features. Additionally, we incorporated positional embeddings into the image sequence to enhance the model’s perception of spatial information, thereby improving the accuracy and robustness of grasp point prediction. Extensive simulations demonstrate that VM-Grasp achieves competitive results compared to existing grasping approaches, achieving higher accuracies of 98.9%, 94.6%, and 96.6% on the Cornell, Jacquard, and Jacquard v2 datasets, respectively. Real-world experiments conducted using a 6DoF myCobot robot have also demonstrated its ability to grasp previously unseen objects in various scenarios.