DeepSweep: Real-Time Multi-View 3D Pose Estimation Via Cross-View Deep Matching and Plane Sweeping
摘要
Existing multi-view multi-person 3D human pose estimation often uses explicit cross-matching methods or implicit 3D voxel and projection to estimate joint positions. Explicit methods are limited by accuracy, while implicit ones affect real-time performance. In this paper, we propose DeepSweep, a novel multi-view 3D pose estimation method based on re-identification (Re-ID) and plane sweep algorithm, which combines the advantages of both explicit and implicit matching to estimate 3D poses. Specifically, DeepSweep first extracts Re-ID features from the detected 2D poses under multiple views through the Re-ID model and performs cross-matching on the features. Then, the 2D poses are projected back to the virtual planes with continuous depth to obtain score matrics. Pose depth is coarse-to-fine regressed using the matching information and these scores. Finally, multi-view information is fused to obtain the corrected 3D poses. Extensive experimental results demonstrate that our method achieves state-of-the-art performance on Campus, Shelf, and CMU Panoptic popular benchmarks. In addition, we enhance the generalization performance of the method proposed in this paper using simulated datasets, enabling it to adapt to different scenarios.