Calibration-Free Multi-view 3D Hand Pose Estimation for XR Cockpit Interactions

Extended Reality (XR) is increasingly integrated into cockpits, where gesture interaction is the fundamental method of immersive human-computer interaction. However, there are three challenges to reliable 3D hand pose estimation in cockpits: (1) restricted hand observability based on egocentric field of view, (2) infeasibility of cockpit motion-induced calibration, and (3) self-occlusion induced by hand-object interaction. To address these challenges, we propose a calibration-free multi-view pipeline that fuses the egocentric XR headset with external third-person cameras. Our framework comprises two explicit modules: the guidance-driven fusion stage mitigates 3D pose ambiguity by constructing a global reference for feature interaction, and the hypothesis-driven alignment stage models extrinsic parameter uncertainty via multi-hypothesis spatial alignment. Extensive experiments on Dex-YCB and HanCo under uncalibrated setups demonstrate that our approach outperforms existing both calibrated and uncalibrated state-of-the-art methods in accuracy and robustness.