CSI-Cro: A Cross-Domain Sign Language Recognition System Based on Dual-Attention Feature Decoupled Network
摘要
Channel State Information (CSI) based sign language recognition technology provides an innovative solution to improve the involvement of hearing impaired people in education and employment. However, its practical application still encounters two key challenges: (I) Existing systems mostly operate effectively only within the source domain; (II) Cross-domain recognition requires extensive pretraining on target domain data, which limits the model’s generalization capability. Moreover, the complex training strategies further raise the threshold for model deployment. To address the above problems, this paper proposes CSI-Cro: A cross-domain CSI sign language recognition system based on Dual-Attention Feature Decoupled Network. CSI-Cro consists of a signal preprocessing module and a recognition module: The preprocessing module converts the raw CSI signal into a high-dimensional spectrogram via time-frequency transforms. The recognition module innovatively employs a feature decoupling mechanism, using parallel domain and sign language feature extractors to decouple the mixed feature into domain-independent sign language representations and domain-dependent representations. An adaptive feature reconstruction layer then generates a highly discriminative fused tensor, ultimately achieving precise sign language classification. The experimental results show that CSI-Cro is able to achieve 96.85% in-domain recognition accuracy, and 92.73% and 91.3% accuracy in cross-position and cross-user test experiments, respectively, which is better than the current CSI cross-domain recognition system.