Precise Hand Gesture Recognition under Hard Visual Environments using Deep Architecture

Hand gesture recognition is a way of capturing and translating the human signs into commands utilizing a visual interface. In this paper, You Only Look Once (Yolo) based deep single-stage convolutional neural network (CNN) is proposed for real-time multi-hand sign recognition under hard visual environments. The Yolo-v2 and Yolo-v3 models are utilized for real-time hand sign recognition. The DarkNet-19 and DarkNet-53 CNN architectures are used as a backbone networks in Yolo-v2 and Yolo-v3 respectively. Three distinct datasets (NUSHP-II, SENZ-3D, and MITI-HD) are used to train and validate the models. The models are evaluated using a test dataset with an IoU range of 0.5 to 0.95. The Yolo-v2 CNN model achieved an average precision value of 99.10% for AP0.5, 93.00% for AP0.75, and 78.30% for AP0.5:0.95 on the MITI-Hand dataset. The Yolo-v3 CNN model achieved an average precision value of 99.18% for AP0.5. The Yolo model with Adam Optimizer performs better com- pared to other optimization approaches. The prediction time obtained by using the Yolo-v2 and Yolo-v3 CNN models is 20 ms and 25 ms, respectively. The proposed Yolo-v3 CNN model efficiently recognizes the multi-hand signs under hard visual environments.