Video desnowing is a critical research topic in the field of computer vision, as snowfall can obscure video content and degrade visual quality. However, due to the diverse scales, shapes, and complex motion patterns of snowflakes, existing video desnowing methods still face challenges when dealing with real-world snowy scenarios. To effectively remove snowflakes and restore high-quality videos under real snowy conditions, we propose a Multi-Scale Video Desnowing Network (MSVDNet), which combines the strengths of Transformer and UNet architectures. Specifically, we design a Multi-Scale Feature Fusion Module (MSFFM) to enhance multi-scale feature perception by integrating encoder features from multiple hierarchical levels. In addition, we propose a Multi-Scale Adaptive Hybrid Attention Module (MSAHAM), which adaptively focuses on local details and global contextual information by combining multi-scale spatial attention and channel attention to recover backgrounds occluded by snowflakes of various scales. Finally, we introduce contrastive learning to narrow the domain gap between desnowed outputs and real-world clean scenes, thereby enhancing the model’s generalization ability. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on the RVSD benchmark dataset and exhibits superior capability in restoring videos in real-world snowy scenarios.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MSVDNet: A Multi-scale Video Desnowing Network for Real World

  • Mengzhen Xue,
  • Gang Zhou,
  • Linghui Ma,
  • Li Zhang,
  • Yanyun Zhou,
  • Zhenhong Jia

摘要

Video desnowing is a critical research topic in the field of computer vision, as snowfall can obscure video content and degrade visual quality. However, due to the diverse scales, shapes, and complex motion patterns of snowflakes, existing video desnowing methods still face challenges when dealing with real-world snowy scenarios. To effectively remove snowflakes and restore high-quality videos under real snowy conditions, we propose a Multi-Scale Video Desnowing Network (MSVDNet), which combines the strengths of Transformer and UNet architectures. Specifically, we design a Multi-Scale Feature Fusion Module (MSFFM) to enhance multi-scale feature perception by integrating encoder features from multiple hierarchical levels. In addition, we propose a Multi-Scale Adaptive Hybrid Attention Module (MSAHAM), which adaptively focuses on local details and global contextual information by combining multi-scale spatial attention and channel attention to recover backgrounds occluded by snowflakes of various scales. Finally, we introduce contrastive learning to narrow the domain gap between desnowed outputs and real-world clean scenes, thereby enhancing the model’s generalization ability. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on the RVSD benchmark dataset and exhibits superior capability in restoring videos in real-world snowy scenarios.