<p>In the case of human pose estimation under extreme weather conditions such as rainy days, due to factors like insufficient lighting and obstruction by raindrops, the image resolution is low and the noise increases. This leads to the loss of high-frequency information such as image details and textures, thus affecting the accuracy of conventional network models. To address this issue, this paper proposes a Multi-Scale Feature Fusion Network (MSFNet). This network uses the High-Resolution Network (HRNet) as the backbone and designs an upsampling module (Resample) that combines image restoration to enhance the ability to extract high-frequency information. Additionally, a Multi-scale convolution block (MSCB) module is proposed for the feature extraction stage to reduce the interference of irrelevant information on keypoint detection and improve the fitting performance of high-frequency information. Furthermore, an edge loss function is designed to enhance the accuracy of keypoint localization. To facilitate a comprehensive study of human pose estimation under rainy conditions, a synthetic dataset, Rain-MPII, was created based on the MPII dataset. Experimental results show that the proposed algorithm achieves accuracy rates of 90.12%, 88.36%, and 87.57% under different rain intensities, demonstrating superior detection performance compared to mainstream methods.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi-scale feature fusion for robust human pose estimation in rainy conditions

  • Cunli Song,
  • Yuxin Xiang,
  • Xuesong Zhang

摘要

In the case of human pose estimation under extreme weather conditions such as rainy days, due to factors like insufficient lighting and obstruction by raindrops, the image resolution is low and the noise increases. This leads to the loss of high-frequency information such as image details and textures, thus affecting the accuracy of conventional network models. To address this issue, this paper proposes a Multi-Scale Feature Fusion Network (MSFNet). This network uses the High-Resolution Network (HRNet) as the backbone and designs an upsampling module (Resample) that combines image restoration to enhance the ability to extract high-frequency information. Additionally, a Multi-scale convolution block (MSCB) module is proposed for the feature extraction stage to reduce the interference of irrelevant information on keypoint detection and improve the fitting performance of high-frequency information. Furthermore, an edge loss function is designed to enhance the accuracy of keypoint localization. To facilitate a comprehensive study of human pose estimation under rainy conditions, a synthetic dataset, Rain-MPII, was created based on the MPII dataset. Experimental results show that the proposed algorithm achieves accuracy rates of 90.12%, 88.36%, and 87.57% under different rain intensities, demonstrating superior detection performance compared to mainstream methods.