Scholarlink

<p>We introduce PPENT, a pre-training pose embedding that refurbishes video-estimated 3D skeletons into motion-captured-technology (MCT) based consistent joints for real-time, word-level sign language recognition (SLR). PPENT takes MediaPipe (MP) estimates, selects 57 joints per frame, and learns a supervised mapping to the motion-captured-aligned joint space using lightweight temporal backbones: axis-wise MLP heads with optional 1D-CNN/LSTM variants. The refurbished sequence is projected to a compact embedding and consumed by either (i) an LSTM classifier trained on reconstructed joints or (ii) a pre-trained skeletal sign classifier built on real 3D data. On a 120-gloss protocol, PPENT reduces pose error by 23% and improves top-1 recognition by 28% over direct recognition on estimated joints, while sustaining real-time throughput on low-cost hardware. Ablations attribute the gains to the MP to MCT refurbishment and axis-wise reconstruction loss. A prototype demonstrates low-latency on-device inference suitable for deployment.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Ppent: a pose embedding refinement framework aligning estimated and motion-captured skeletons for real-time word-level sign language recognition

  • 期刊论文

  • 发表日期：

    2025年12月19日

查看全文
  • P. V. V. Kishore,
  • G. Hima Bindu,
  • B. Prasad,
  • D. Anil Kumar,
  • P. Praveen Kumar,
  • M. Suneetha,
  • P. Sasikiran,
  • E. Kiran Kumar

摘要

We introduce PPENT, a pre-training pose embedding that refurbishes video-estimated 3D skeletons into motion-captured-technology (MCT) based consistent joints for real-time, word-level sign language recognition (SLR). PPENT takes MediaPipe (MP) estimates, selects 57 joints per frame, and learns a supervised mapping to the motion-captured-aligned joint space using lightweight temporal backbones: axis-wise MLP heads with optional 1D-CNN/LSTM variants. The refurbished sequence is projected to a compact embedding and consumed by either (i) an LSTM classifier trained on reconstructed joints or (ii) a pre-trained skeletal sign classifier built on real 3D data. On a 120-gloss protocol, PPENT reduces pose error by 23% and improves top-1 recognition by 28% over direct recognition on estimated joints, while sustaining real-time throughput on low-cost hardware. Ablations attribute the gains to the MP to MCT refurbishment and axis-wise reconstruction loss. A prototype demonstrates low-latency on-device inference suitable for deployment.