End-to-End mmWave-Based Human Pose Estimation from Raw Signal

This paper proposes a novel method for extracting human pose from raw millimeter-wave (mmWave) radar signals. Traditional mmWave radar human pose estimation techniques typically involve a two-step process: a predefined preprocessing module with discrete Fourier transform (DFT) and a pose estimation network, which first extracts motion-related range, velocity and angle cues from the raw radar signals and then reconstructs the pose from them. However, such a pipeline suffers from two drawbacks. On one hand, the transformation bases of DFT is preset, which could lead to poor adaptability to different poses. On the other hand, the resolution of the extracted information from radar data is uniform, resulting in weakning crucial pose-related features. To address this problem, we unify these two steps by using end-to-end network that employs a DFT-initialized learnable preprocessing module and a pose-specific downstream network to directly reconstruct the human skeleton from the raw signals. Experimental results on the self-collected human motion dataset validate the effectiveness of the proposed method.