Gesture Synthesis for Sign Language Using Generative Adversarial Networks
摘要
Sign language generation and recognition are essential for improving accessibility for the deaf and hard-of-hearing community. This study explores deep learning techniques, combining skeleton-based models and StackGAN to synthesize high-quality Indian Sign Language (ISL) representations. Skeletal structures are first extracted to capture hand and finger movements, though they lack fine details like finger shapes and facial expressions. To address this, a two-stage StackGAN refines low-resolution skeletons into realistic ISL signs. The model, evaluated using qualitative and quantitative metrics, effectively synthesizes ISL alphabets and numbers, with generator and discriminator losses of 47% and 51%, respectively. While the results closely resemble real ISL gestures, minor distortions remain due to limited data. Future work will expand the model to word-level sequences, dynamic gestures, and multimodal learning for real-time applications, laying the groundwork for robust ISL communication systems.