Scholarlink

<p>Transformer architectures have gained prominence in robotic slip detection owing to their capacity in capturing spatio-temporal correlations within multimodal sensory streams. Nevertheless, the conventional full-token self-attention mechanism inherent in standard Transformers introduces certain limitations, including quadratic computational complexity, data-hungry training requirements. To address these limitations, we propose a token reorganization scheme, leveraging class token attention saliency in the deep layers of the neural network to dynamically reorganize tokens. This approach selectively retains task-critical information in the visual-tactile modalities, while efficiently compressing background tokens through adaptive averaging, thereby optimizing computational efficiency without compromising performance. This scheme reduces the computational complexity of salient self-attention. Based on this scheme, a visual-tactile fusion Transformer for robot slip detection is developed. Extensive experiments have shown that the proposed Transformer can achieve state-of-the-art performance, which will reduce the computational cost, while increasing the detection accuracy to 86.75% by enhancing the focus on deformation-sensitive features. This work provides an effective visual-tactile perception solution for robust robot manipulation under real-world uncertainties. Our code and dataset are available at <a href="https://github.com/Bugs-Bunny01/VTF-AVIT">https://github.com/Bugs-Bunny01/VTF-AVIT</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Efficient visual-tactile transformer with token reorganization for robotic slip detection

  • 期刊论文

  • 发表日期：

    2025年4月28日

查看全文
  • Liu Chaowen,
  • Wu Hao,
  • Shen Jingjin

摘要

Transformer architectures have gained prominence in robotic slip detection owing to their capacity in capturing spatio-temporal correlations within multimodal sensory streams. Nevertheless, the conventional full-token self-attention mechanism inherent in standard Transformers introduces certain limitations, including quadratic computational complexity, data-hungry training requirements. To address these limitations, we propose a token reorganization scheme, leveraging class token attention saliency in the deep layers of the neural network to dynamically reorganize tokens. This approach selectively retains task-critical information in the visual-tactile modalities, while efficiently compressing background tokens through adaptive averaging, thereby optimizing computational efficiency without compromising performance. This scheme reduces the computational complexity of salient self-attention. Based on this scheme, a visual-tactile fusion Transformer for robot slip detection is developed. Extensive experiments have shown that the proposed Transformer can achieve state-of-the-art performance, which will reduce the computational cost, while increasing the detection accuracy to 86.75% by enhancing the focus on deformation-sensitive features. This work provides an effective visual-tactile perception solution for robust robot manipulation under real-world uncertainties. Our code and dataset are available at https://github.com/Bugs-Bunny01/VTF-AVIT.