Sign language translation plays a critical role in bridging the gap between the deaf and hard-of-hearing community and the hearing world. However, this task remains highly challenging due to the scarcity of annotated resources, linguistic ambiguity, and the prevalence of compound glosses in sign language annotations. To address these challenges, we propose G2TTA-GAS, an enhanced Gloss2Text Transformer architecture that integrates the Gloss Annotation Scheme to generate normalized and enriched gloss inputs. The scheme systematically decomposes compound glosses into semantically meaningful smaller units while preserving essential linguistic features. This structured representation facilitates better alignment with natural language outputs and strengthens the model’s ability to learn context-independent representations. Experimental evaluations on the ASLG-PC12 dataset demonstrate that G2TTA-GAS achieves an average relative improvement of 3.63% in BLEU-4 on the training set and 3.04% on the test set compared to G2TTA-BERT. Moreover, G2TTA-GAS substantially improves efficiency, reducing inference latency by 49.92% relative to G2TTA and by 55.85% relative to G2TTA-BERT.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Gloss Annotation Scheme for Enhanced Sign Language Translation

  • Sam Nguyen-Xuan,
  • Giap Dang Le,
  • Han Nguyen

摘要

Sign language translation plays a critical role in bridging the gap between the deaf and hard-of-hearing community and the hearing world. However, this task remains highly challenging due to the scarcity of annotated resources, linguistic ambiguity, and the prevalence of compound glosses in sign language annotations. To address these challenges, we propose G2TTA-GAS, an enhanced Gloss2Text Transformer architecture that integrates the Gloss Annotation Scheme to generate normalized and enriched gloss inputs. The scheme systematically decomposes compound glosses into semantically meaningful smaller units while preserving essential linguistic features. This structured representation facilitates better alignment with natural language outputs and strengthens the model’s ability to learn context-independent representations. Experimental evaluations on the ASLG-PC12 dataset demonstrate that G2TTA-GAS achieves an average relative improvement of 3.63% in BLEU-4 on the training set and 3.04% on the test set compared to G2TTA-BERT. Moreover, G2TTA-GAS substantially improves efficiency, reducing inference latency by 49.92% relative to G2TTA and by 55.85% relative to G2TTA-BERT.