<p>Convolution-based neural networks have been extensively employed in motion deblurring. However, two issues prevent them from realizing their full potential: (1) limited receptive fields owing to the kernel size; (2) failure to adequately characterize the complex and varying degrees of blur in the inference stage, owing to static weights. Although self-attention can alleviate these issues, its computational complexity is prohibitively high, as it scales with the square of the length of the image token sequence. Therefore, we propose a hybrid convolution and dimension-based self-attention method for motion deblurring, which efficiently combines the advantages of convolution and self-attention. First, a dimension-based self-attention module focusing on the macro-scale blur patterns from row, column, and channel dimensions is introduced to compute long-range attention at low computational cost. Second, a DL-Conv block consisting of depth-wise separable convolution with a large kernel size and layer normalization is employed to extract micro-scale local blur features with a larger receptive field. The complementarity between micro-scale local blur features and macro-scale blur patterns enables comprehensive characterization of the blur information. Finally, a complementary loss function that considers different aspects, including pixel-level loss, frequency-domain loss, and structural similarity loss is adopted to train the model. Extensive experimental results demonstrate that the proposed model outperforms other state-of-the-art (SOTA) methods. Compared with other self-attention-based methods, the proposed model achieves better performance with fewer parameters and shorter inference time.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

HCDFormer: A Hybrid Convolution and Dimension-based Self-attention Method for Image Motion Deblurring

  • Weizhi Yang,
  • Yi Xie,
  • Jiangqun Ni,
  • Ying Gao,
  • Jun Tang

摘要

Convolution-based neural networks have been extensively employed in motion deblurring. However, two issues prevent them from realizing their full potential: (1) limited receptive fields owing to the kernel size; (2) failure to adequately characterize the complex and varying degrees of blur in the inference stage, owing to static weights. Although self-attention can alleviate these issues, its computational complexity is prohibitively high, as it scales with the square of the length of the image token sequence. Therefore, we propose a hybrid convolution and dimension-based self-attention method for motion deblurring, which efficiently combines the advantages of convolution and self-attention. First, a dimension-based self-attention module focusing on the macro-scale blur patterns from row, column, and channel dimensions is introduced to compute long-range attention at low computational cost. Second, a DL-Conv block consisting of depth-wise separable convolution with a large kernel size and layer normalization is employed to extract micro-scale local blur features with a larger receptive field. The complementarity between micro-scale local blur features and macro-scale blur patterns enables comprehensive characterization of the blur information. Finally, a complementary loss function that considers different aspects, including pixel-level loss, frequency-domain loss, and structural similarity loss is adopted to train the model. Extensive experimental results demonstrate that the proposed model outperforms other state-of-the-art (SOTA) methods. Compared with other self-attention-based methods, the proposed model achieves better performance with fewer parameters and shorter inference time.