Identifying Hand Pose Used in Sign Language Using Key-Point and Transfer Learning Technique

Sign language recognition is now possible because of the advancements in processors, computer vision, image processing, machine and deep learning techniques. This development has made it feasible to interpret sign languages effectively by using cutting-edge technology. It has made communication possible for those with speech and hearing impairments. This will help them improve the quality of their lives when they need to connect with rest of the world. For the said cause, we have suggested a methodology. To extract the features, hand key points and transfer learning techniques with vgg16 model have been used. To prove its efficacy, we worked on three different datasets: (i) NUS hand posture datasets II (ii) ISL dataset from Kaggle and (iii) our own ISL dataset created with the help of multiple users in diverse backgrounds. Our methodology works efficiently on all three datasets and exhibits validation accuracy of 99.12, 99.89 and 99.78 respectively. Result demonstrates that it works effectively with various images and mentioned diverse datasets. Hence, we feel this approach is generalizable and can be effectively useful with different sign languages.