Advanced Custom Gesture Recognition for Needle Manipulation in Mixed Reality Acupuncture Training

Reliable gesture recognition is essential for mixed reality acupuncture training where fine motor actions must be detected in real time. We present a training-free, rule-based recognizer that operates on OpenXR-standardized hand joint streams and uses anatomy-aware, scale-normalized geometric features. The module detects four core acupuncture gestures: two-finger pinching, three-finger pinching, four-finger pinching, and tail pressing; and two action sequences: lift-thrusting and twirling. Evaluation on a remapped public dataset and a user study with twenty students shows consistently high accuracy for per-trial gestures and per-cycle sequences. On the public dataset, performance remained stable under viewpoint and distance variation. The recognizer is designed to be portable across OpenXR devices, and occlusion, field-of-view limits, and bimanual synchrony remain the primary causes of unrecognition.