Neural machine translation techniques for English text to Pakistan sign language gloss translation

This study aims to develop a neural machine translation system for converting English text into Pakistan Sign Language (PSL) gloss. It seeks to address the challenges faced by the deaf community in Pakistan in acquiring literacy skills due to the scarcity of resources for PSL translation. The research reviewed various datasets and translation models, emphasizing the need for specialized systems for PSL. It introduced the first open-source English-to-PSL sentence-level dataset and explored the application of deep learning models, focusing on the performance of BART and T5 and other deep learning in translating English to PSL gloss through transfer learning by first training on American Sign Language (ASL) and transfer learning to PSL. The study found that deep learning models, particularly BART and T5, showed promising results in translating English text into ASL and PSL gloss achieving the best BLEU-4 accuracy on the ASLGPC-12 dataset of 0.86 and BLEU-1 accuracy of 0.48 on the PSL Dataset. The research contributed a comprehensive dataset and advanced translation models for PSL, aiming to enhance accessibility for the deaf community in Pakistan. The research underscores the potential of modern transformer architectures in sign language translation. By providing a substantial dataset and demonstrating the effectiveness of BART and T5 small models, this work represents a significant step forward in improving accessibility for the deaf community. Future work will extend these findings to longer texts and increase the diversity of the PSL dataset.