Computer vision has made significant advancements across various domains; however, occlusion remains a major challenge, particularly in pose estimation tasks. While several techniques have been introduced to address complex backgrounds, self-occlusion, scale variation, and overlapping individuals, a less explored yet impactful problem is occlusion caused by out-of-frame scenarios. This study introduces this type of occlusion as a critical area for improving the accuracy, efficiency, and effectiveness of pose estimation models in real-world applications. To investigate this, a custom dataset comprising 4,410 images was developed, featuring individuals performing push-ups, sit-ups, and squats. Data augmentation and a masked scripting method were applied to simulate out-of-frame occlusion. The model was trained using the Roboflow platform, with YOLOv8 employed for multi-object detection and DeepSORT for individual ID tracking. To mitigate occlusion-related keypoint loss, a Kalman filter was used to predict missing keypoints due to motion, while a Graph Convolutional Network (GCN) was implemented to infer keypoints lost in out-of-frame scenarios. The model achieved a Mean Average Precision (mAP@50) of 98.71%, with mAP@95 scores of 86.16% and 85.34%. For occluded individuals, the Mean Per Joint Position Error (MPJPE) was 5.45 mm for squats, 9.37 mm for push-ups, and 4.22 mm for sit-ups. In comparison, non-occluded cases showed MPJPE values of 9.53 mm (squats), 6.08 mm (push-ups), and 5.83 mm (sit-ups). Despite the higher MPJPE in occluded scenarios, the Percentage of Correct Keypoints (PCK) remained high, indicating reliable keypoint detection. Interestingly, while non-occluded cases had lower MPJPE, they exhibited comparatively lower PCK scores. These findings suggest that the GCN model effectively infers missing keypoints under occlusion, maintaining overall detection consistency, while YOLOv8 performs well under full visibility but may struggle with keypoint completeness. These findings highlight how occlusion affects keypoint detection differently and emphasize the role of model robustness, dataset diversity, and movement dynamics in addressing such challenges.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Out-of-Frame Occlusion Handling in 3D Pose Estimation for Multi-view and Multi-person

  • King Miles Edrianne A. Ramos,
  • Jheanel E. Estrada

摘要

Computer vision has made significant advancements across various domains; however, occlusion remains a major challenge, particularly in pose estimation tasks. While several techniques have been introduced to address complex backgrounds, self-occlusion, scale variation, and overlapping individuals, a less explored yet impactful problem is occlusion caused by out-of-frame scenarios. This study introduces this type of occlusion as a critical area for improving the accuracy, efficiency, and effectiveness of pose estimation models in real-world applications. To investigate this, a custom dataset comprising 4,410 images was developed, featuring individuals performing push-ups, sit-ups, and squats. Data augmentation and a masked scripting method were applied to simulate out-of-frame occlusion. The model was trained using the Roboflow platform, with YOLOv8 employed for multi-object detection and DeepSORT for individual ID tracking. To mitigate occlusion-related keypoint loss, a Kalman filter was used to predict missing keypoints due to motion, while a Graph Convolutional Network (GCN) was implemented to infer keypoints lost in out-of-frame scenarios. The model achieved a Mean Average Precision (mAP@50) of 98.71%, with mAP@95 scores of 86.16% and 85.34%. For occluded individuals, the Mean Per Joint Position Error (MPJPE) was 5.45 mm for squats, 9.37 mm for push-ups, and 4.22 mm for sit-ups. In comparison, non-occluded cases showed MPJPE values of 9.53 mm (squats), 6.08 mm (push-ups), and 5.83 mm (sit-ups). Despite the higher MPJPE in occluded scenarios, the Percentage of Correct Keypoints (PCK) remained high, indicating reliable keypoint detection. Interestingly, while non-occluded cases had lower MPJPE, they exhibited comparatively lower PCK scores. These findings suggest that the GCN model effectively infers missing keypoints under occlusion, maintaining overall detection consistency, while YOLOv8 performs well under full visibility but may struggle with keypoint completeness. These findings highlight how occlusion affects keypoint detection differently and emphasize the role of model robustness, dataset diversity, and movement dynamics in addressing such challenges.