Collecting and Analyzing the Mid-Air Gestures Data in Augmented Reality and User Preferences in Closed Elicitation Study

For users of AR-HMDs (i.e., AR glasses), one option for interaction with the HMD is through midair gestures. This modality allows users to move their hands to grab objects, move objects, select menus, and interact with a multiple set of features provided by AR-HMDs. However, midair gestures are still difficult to recognize. For example, the Microsoft HoloLens 2 only provides a few gestures, and there are many false positives in the gesture recognition of the system. Further, it is not clear what gestures the users prefer. As a result, we performed a study with data collection on midair gestures for AR-HMDs with the aim of providing a gesture dataset that can be utilized in machine learning models. In addition, we analyzed the user gesture preferences for object manipulation. Our collected dataset includes the trajectory of hand gestures and eye gaze, which are captured by Articulated Hand Tracking (AHAT), inertial measurement unit (IMU) and other sensors. Our analysis showed that users have different gesture preferences for different categories of referents.