Scholarlink

<p>The lack of scalable automatic sign language recognition systems remains a critical barrier for hearing-impaired communication, as current methods rely on computationally intensive video models that are difficult to scale and deploy in real-world scenarios, and often exhibit limited generalization across different sign languages. To address these issues, we present DSTAdapter, a parameter-efficient transfer learning framework that adapts frozen CLIP models for video-based sign language recognition through spatial-temporal adaptation. Our solution introduces three key innovations: a decoupled 3D convolution design separating temporal and spatial processing, a dynamic channel-aware feature fusion module, and a lightweight framework for efficient resource-constrained deployment. Extensive experiments on four benchmark datasets demonstrate that DSTAdapter achieves state-of-the-art performance, while requiring only 4% of tunable parameters compared to full fine-tuning. Notably, it reduces training time by 30% and GPU memory consumption by 60% compared to full fine-tuning approaches on the Bukva dataset. These advances not only establish lightweight technical standards for efficient video understanding, but also enable practical applications of assistive technologies for hearing-impaired communities. The code of this work is available at <a href="https://github.com/BLOOM0-0/DSTAdapter">https://github.com/BLOOM0-0/DSTAdapter</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Dstadapter: divided spatial-temporal adapter fine-tuning method for sign language recognition

  • 期刊论文

  • 发表日期：

    2026年3月21日

查看全文
  • Qiuhong Tian,
  • Yijie Yang,
  • Bin Chen,
  • Jiacheng Chen,
  • Junxiao Ning,
  • Lizao Zhang

摘要

The lack of scalable automatic sign language recognition systems remains a critical barrier for hearing-impaired communication, as current methods rely on computationally intensive video models that are difficult to scale and deploy in real-world scenarios, and often exhibit limited generalization across different sign languages. To address these issues, we present DSTAdapter, a parameter-efficient transfer learning framework that adapts frozen CLIP models for video-based sign language recognition through spatial-temporal adaptation. Our solution introduces three key innovations: a decoupled 3D convolution design separating temporal and spatial processing, a dynamic channel-aware feature fusion module, and a lightweight framework for efficient resource-constrained deployment. Extensive experiments on four benchmark datasets demonstrate that DSTAdapter achieves state-of-the-art performance, while requiring only 4% of tunable parameters compared to full fine-tuning. Notably, it reduces training time by 30% and GPU memory consumption by 60% compared to full fine-tuning approaches on the Bukva dataset. These advances not only establish lightweight technical standards for efficient video understanding, but also enable practical applications of assistive technologies for hearing-impaired communities. The code of this work is available at https://github.com/BLOOM0-0/DSTAdapter.