Scholarlink

<p>3D hand mesh reconstruction from monocular images remains challenging due to frequent occlusions caused by hand–object interactions. Existing methods often neglect the joint hand–object interaction (HOI) relationship, leading to suboptimal results. To address this issue, we propose InteractHand, a novel framework that explicitly incorporates interaction knowledge into the reconstruction process. Our method first segments the hand region from the input image, combining it with the original HOI image to extract both primary hand and interaction information. A learnable contextual gating block (CGBlock) is designed to emphasize the hand region, particularly around the interacting boundaries. The fused hand features are then refined through a local self-enhancement transformer (LET) with cross-shaped attention, capturing both horizontal and vertical context to enhance local details. By integrating HOI information with accurate hand segmentation, InteractHand achieves state-of-the-art performance on the DexYCB and HO3D datasets, demonstrating its effectiveness in reconstructing 3D hand meshes, especially in scenes with significant occlusions. Our implementation will be open source. Code is available at <a href="https://github.com/wang-yan-adrian/InteractHand">https://github.com/wang-yan-adrian/InteractHand</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Interacthand: Robust 3D Hand Mesh Reconstruction via Interaction-Aware Segmentation and Refinement

  • 期刊论文

  • 发表日期：

    2025年6月29日

查看全文
  • Yan Wang,
  • Mingwen Shao,
  • Chao Wang,
  • Kai Xu,
  • Xiaolin Lu

摘要

3D hand mesh reconstruction from monocular images remains challenging due to frequent occlusions caused by hand–object interactions. Existing methods often neglect the joint hand–object interaction (HOI) relationship, leading to suboptimal results. To address this issue, we propose InteractHand, a novel framework that explicitly incorporates interaction knowledge into the reconstruction process. Our method first segments the hand region from the input image, combining it with the original HOI image to extract both primary hand and interaction information. A learnable contextual gating block (CGBlock) is designed to emphasize the hand region, particularly around the interacting boundaries. The fused hand features are then refined through a local self-enhancement transformer (LET) with cross-shaped attention, capturing both horizontal and vertical context to enhance local details. By integrating HOI information with accurate hand segmentation, InteractHand achieves state-of-the-art performance on the DexYCB and HO3D datasets, demonstrating its effectiveness in reconstructing 3D hand meshes, especially in scenes with significant occlusions. Our implementation will be open source. Code is available at https://github.com/wang-yan-adrian/InteractHand.