Image deraining in complex rainy scenes remains a significant challenge due to the difficulty of disentangling rain streaks from background content and the lack of effective global context modeling. In this paper, we propose a novel Mamba-Transformer hybrid deraining framework for single image deraining. The encoder leverages Mamba modules for hierarchical feature extraction and efficient global representation, while the decoder incorporates Transformer-based attention mechanisms to enhance feature reconstruction. To overcome the unidirectional limitations of traditional state-space models, we introduce the Omni Selective Scanning (OSS) mechanism, which performs parallel spatial scans along six directions. To effectively remove raindrops that produce high-intensity frequency components in specific orientations, we integrate frequency-domain processing with state-space models and attention mechanisms. Specifically, the Frequency-Enhanced Omni Selective Scanning (FOSS) block in the encoder combines the fast Fourier transform with state-space modeling to isolate rain components in the frequency domain. In the decoder, the Spectral Band Self-Attention (SBSA) module suppresses low-frequency rain streaks, while the Spectral Enhancement Feed-Forward (SEFF) module refines feature representations. Extensive experiments on multiple datasets demonstrate that our framework significantly outperforms state-of-the-art methods in rain removal, while maintaining superior performance in diverse rainy conditions and preserving fine-grained scene details.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Hybrid Mamba-Transformer with Frequency Enhancement for Single Image Deraining

  • Yue Que,
  • Wenjun Xia,
  • Xue Xia,
  • Jie You

摘要

Image deraining in complex rainy scenes remains a significant challenge due to the difficulty of disentangling rain streaks from background content and the lack of effective global context modeling. In this paper, we propose a novel Mamba-Transformer hybrid deraining framework for single image deraining. The encoder leverages Mamba modules for hierarchical feature extraction and efficient global representation, while the decoder incorporates Transformer-based attention mechanisms to enhance feature reconstruction. To overcome the unidirectional limitations of traditional state-space models, we introduce the Omni Selective Scanning (OSS) mechanism, which performs parallel spatial scans along six directions. To effectively remove raindrops that produce high-intensity frequency components in specific orientations, we integrate frequency-domain processing with state-space models and attention mechanisms. Specifically, the Frequency-Enhanced Omni Selective Scanning (FOSS) block in the encoder combines the fast Fourier transform with state-space modeling to isolate rain components in the frequency domain. In the decoder, the Spectral Band Self-Attention (SBSA) module suppresses low-frequency rain streaks, while the Spectral Enhancement Feed-Forward (SEFF) module refines feature representations. Extensive experiments on multiple datasets demonstrate that our framework significantly outperforms state-of-the-art methods in rain removal, while maintaining superior performance in diverse rainy conditions and preserving fine-grained scene details.