Recent advancements in convolutional neural networks (CNNs) and vision transformers have greatly improved hand pose estimation from monocular RGB images. However, existing accurate 2D hand pose estimation methods often suffer from large parameter sizes, high model complexity, and poor real-time performance, making them unsuitable for portable devices such as mobile phones. In this paper, we introduce a lightweight network specifically designed for accurate 2D hand pose estimation. At the core of our approach is Lite-MobileViT, a newly designed encoder block inspired by MobileViT(Mehta & Rastegari, 2022). Additionally, we propose a novel feature enhancement stem block, Coordinate and Polarized Self-Attention (CPSA), which extracts richer spatial feature information from input images. With a reduced parameter count of only 1.48 million, our network demonstrates superior efficiency. Evaluations on the widely recognized RHD (Rendered Hand Pose datasets) and CMU (CMU Panoptic Dataset) datasets show that our model not only has fewer parameters than mainstream 2D hand pose estimation networks, but also achieves comparable or superior performance, making it well-suited for real-time applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Feature-Enhanced Lightweight Network for Accurate 2D Hand Pose Estimation

  • Bin Zhang,
  • Yong Gu

摘要

Recent advancements in convolutional neural networks (CNNs) and vision transformers have greatly improved hand pose estimation from monocular RGB images. However, existing accurate 2D hand pose estimation methods often suffer from large parameter sizes, high model complexity, and poor real-time performance, making them unsuitable for portable devices such as mobile phones. In this paper, we introduce a lightweight network specifically designed for accurate 2D hand pose estimation. At the core of our approach is Lite-MobileViT, a newly designed encoder block inspired by MobileViT(Mehta & Rastegari, 2022). Additionally, we propose a novel feature enhancement stem block, Coordinate and Polarized Self-Attention (CPSA), which extracts richer spatial feature information from input images. With a reduced parameter count of only 1.48 million, our network demonstrates superior efficiency. Evaluations on the widely recognized RHD (Rendered Hand Pose datasets) and CMU (CMU Panoptic Dataset) datasets show that our model not only has fewer parameters than mainstream 2D hand pose estimation networks, but also achieves comparable or superior performance, making it well-suited for real-time applications.