Hand Gesture Recognition and Text-to-Gesture Generation System Using VGG16

Gesture-based communication, particularly in American Sign Language (ASL), is a fundamental aspect of human interaction. In recent years, advancements in deep learning have paved the way for automated hand gesture recognition and text-to-gesture generation systems. This paper presents a novel approach utilizing the VGG16 and InceptionV3 architectures for accurate hand gesture recognition and the generation of corresponding gestures from textual descriptions, with a focus on ASL. The VGG16 model is fine-tuned to recognize intricate ASL hand gestures, while InceptionV3 is adapted to comprehend textual ASL descriptions. The extracted features are then fed into a multimodal fusion network for joint understanding and integration of image and ASL text information. The model is fine-tuned using transfer learning techniques to optimize its performance for ASL gesture classification. On the other hand, for text-to-ASL gesture generation, a dataset containing text-ASL gesture pairs is employed to train the InceptionV3 model to associate textual ASL descriptions with corresponding gestures.