Gesture Based User Interface Access Using Convolution Neural Network
摘要
Gesture-based UI Interaction is a process that facilitates user-system interaction without the need for physical touching. This method allows users to input characters or numbers and execute corresponding actions through gestures. The system displays symbols representing specific characters or numbers, and by presenting a particular action symbol, the associated action is performed. Particularly during viral crises, there is a considerable demand for touchless systems that prioritize minimal physical contact, thereby reducing transmission risks, promoting hygiene practices, and upholding public confidence. The implementation of touchless systems during viral crises involves the utilization of a Convolutional Neural Network (CNN) algorithm that enables touchless detection and interpretation of hand gestures performed by the user. The CNN accurately identifies hand gestures and translates them into characters or actions, with training phase accuracies of 0.912 for alphabet, 0.922 for numeric, and 0.933 for actions datasets, and testing phase accuracies of 0.924 for alphabet, 0.941 for numeric, and 0.953 for actions datasets. The touchless system achieved a combined accuracy of 0.922 during training and 0.939 during testing.