VisualSign: Revolutionizing Video Accessibility Through Sign Language Translation
摘要
In the digital age, equitable access to educational content is essential for all individuals, including those who are deaf or hard of hearing. Traditional educational videos often pose significant accessibility challenges for these audiences. This project presents a solution through the development of an automated system that generates sign language interpretations for educational videos. By employing advanced text extraction and Natural Language Processing (NLP) techniques, including state-of-the-art transformer models such as BERT (Bidirectional Encoder Representations from Transformers) and T5 (Text-To-Text Transfer Transformer), the system extracts and processes text from video content. It then translates English syntax into sign language grammar, culminating in the creation of sign language videos based on the generated sign gloss. This comprehensive methodology encompasses the extraction process, translation mechanisms, sign language video generation, and the integration of relevant technologies. The project also investigates the use of Sign Gesture Markup Language (SiGML) for standardized representation of sign language gestures and expressions, ensuring interoperability with existing resources. The proposed work findings highlights the system’s potential to enhance accessibility and inclusivity within educational environments, ultimately fostering a more inclusive learning experience for all learners.