Reconstructing Challenging Hand Posture from Multi-modal Input

3D Hand reconstruction is critical for immersive VR/AR, action understanding or human healthcare. Without considering actual skin or texture details, existing solutions have concentrated on recovering hand pose and shape using parametric models or learning techniques. In this study, we introduce a challenging hand dataset, CHANDS, which is composed of articulated precise 3D geometry corresponding to previously unheard-of challenging gestures performed by real hands. Specifically, we construct a multi-view camera setup to acquire multi-view images for initial 3D reconstructions and use a hand tracker to separately capture the skeleton. Then, we present a robust method for reconstructing an articulated geometry and matching the skeleton to the geometry using a template. In addition, we build a hand pose model from CHANDS that covers a wider range of poses and is particularly helpful for difficult poses.