In recent years, single-view hand mesh reconstruction has seen significant advancements. While existing methods perform well on motion-capture datasets, the lack of accurately annotated in-the-wild datasets remains a critical challenge for supervised 3D hand mesh reconstruction. Additionally, due to domain gaps, directly applying existing models to in-the-wild datasets has yielded unsatisfactory results. To address these issues, we propose UnDAHand, which aims to transfer knowledge learned from motion-capture source domains to in-the-wild target domains in an unsupervised manner. Specifically, to bridge the domain gap between different datasets, we introduce a fine-to-coarse pseudo label update strategy, which utilizes a “teacher-for-teacher” correction process for generating pseudo labels. Furthermore, recognizing the importance of consistency learning and the diversity of data in enhancing model transferability and generalization, we develop an augmentation consistency learning module to capture more robust data representations. Experimental results demonstrate the effectiveness of our method in domain adaptation across various datasets and its robustness when applied to unlabeled in-the-wild data.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Unsupervised Domain Adaptive Hand Mesh Reconstruction of 2D Images in the Wild

  • Xinyi Hou,
  • Huayi Zhou,
  • Yue Ding,
  • Hongtao Lu

摘要

In recent years, single-view hand mesh reconstruction has seen significant advancements. While existing methods perform well on motion-capture datasets, the lack of accurately annotated in-the-wild datasets remains a critical challenge for supervised 3D hand mesh reconstruction. Additionally, due to domain gaps, directly applying existing models to in-the-wild datasets has yielded unsatisfactory results. To address these issues, we propose UnDAHand, which aims to transfer knowledge learned from motion-capture source domains to in-the-wild target domains in an unsupervised manner. Specifically, to bridge the domain gap between different datasets, we introduce a fine-to-coarse pseudo label update strategy, which utilizes a “teacher-for-teacher” correction process for generating pseudo labels. Furthermore, recognizing the importance of consistency learning and the diversity of data in enhancing model transferability and generalization, we develop an augmentation consistency learning module to capture more robust data representations. Experimental results demonstrate the effectiveness of our method in domain adaptation across various datasets and its robustness when applied to unlabeled in-the-wild data.