Bridging the Physical and Virtual Worlds: A Hand Tracking Gesture Recognition System for XR Applications

Extended reality (XR) is a term that encompasses augmented reality (AR), virtual reality (VR), and mixed reality (MR). These technologies allow users to interact with digital content in immersive and realistic ways. One of the challenges of XR is to provide natural and intuitive ways to control and manipulate the virtual environment. This paper presents the development of a gesture recognition system for hand tracking in extended reality technologies (XR) and its practical applications. The system uses a gesture definition language that allows the user to create custom gestures based on the hand skeleton model. The system was implemented using the Oculus Quest 2 device and the Oculus SDK in the Unity game engine and is compatible with all devices that support hand tracking. The gesture recognition system was implemented using a three-bone thumb and little finger model that is compatible with all hand models that contain four bones. The implementation of this gesture recognition system in XR has several benefits, including increased immersion, improved interaction with virtual objects, and reduced physical fatigue compared to traditional input methods such as controllers or keyboards. This system provides users with a more intuitive and natural way to interact with their virtual environment. Overall, this work demonstrates the feasibility and potential of integrating gesture recognition into XR technologies and its ability to enhance user experience and interaction. The system presented in this work can serve as a foundation for future developments in this area, with potential applications in gaming, education, healthcare, and other fields.