3D Hand Gesture Reconstruction in Occluded Environments Using a Transformer-Based Approach
摘要
The task of 3D hand gesture reconstruction from a single viewpoint is inherently challenging due to the frequent occurrence of occlusions that can obscure critical joint information. Traditional methods rely heavily on complete visual inputs, which are often unavailable in real-world scenarios. This paper introduces a novel transformer-based approach to 3D hand gesture reconstruction that effectively addresses the issue of occlusions. The proposed method leverages the transformer architecture to model both temporal and spatial dependencies and incorporates a masking operation to simulate occlusion scenarios. By simulating occlusions during training, the model learns to infer missing joint information effectively. Additionally, the model integrates anatomical prior knowledge of hand gestures to guide the prediction of occluded joint positions, enhancing the model’s performance in complex environments.