<p>Currently, most deep learning-based super-resolution techniques primarily operate in the spatial domain, utilizing similar methods to process high- and low-frequency information in images. However, this often results in edge blurring. To address this issue, this paper introduces a novel structure that integrates wavelet transform and transformer mechanisms. The proposed method effectively segregates high- and low-frequency image information via discrete wavelet transform (DWT) and learns their correlations through a self-attention mechanism to enhance super-resolution outcomes. Specifically, the input image/feature is decomposed into four frequency domain components using DWT, which are concatenated to form a full-frequency domain feature map. A high-frequency feature map is constructed from three of these components. A new feature map is then generated using multi-head self-attention, with the full-frequency domain feature map serving as the query and value, and the high-frequency feature map as the key. The output feature map is produced by applying inverse DWT, with the new feature map serving as the low-frequency component and the original high-frequency components retained. Additionally, a parallel 1 × 1 convolution filter is employed to minimize information loss. Furthermore, a super-resolution network for remote sensing images is constructed by combining wavelet transform and transformer, incorporating hierarchical residual connections to enable the network to focus on learning high-frequency information. Experimental results on a publicly available remote sensing dataset demonstrate the superiority of the proposed method compared to existing approaches.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

WTT: combining wavelet transform with transformer for remote sensing image super-resolution

  • Jingyi Liu,
  • Xiaomin Yang

摘要

Currently, most deep learning-based super-resolution techniques primarily operate in the spatial domain, utilizing similar methods to process high- and low-frequency information in images. However, this often results in edge blurring. To address this issue, this paper introduces a novel structure that integrates wavelet transform and transformer mechanisms. The proposed method effectively segregates high- and low-frequency image information via discrete wavelet transform (DWT) and learns their correlations through a self-attention mechanism to enhance super-resolution outcomes. Specifically, the input image/feature is decomposed into four frequency domain components using DWT, which are concatenated to form a full-frequency domain feature map. A high-frequency feature map is constructed from three of these components. A new feature map is then generated using multi-head self-attention, with the full-frequency domain feature map serving as the query and value, and the high-frequency feature map as the key. The output feature map is produced by applying inverse DWT, with the new feature map serving as the low-frequency component and the original high-frequency components retained. Additionally, a parallel 1 × 1 convolution filter is employed to minimize information loss. Furthermore, a super-resolution network for remote sensing images is constructed by combining wavelet transform and transformer, incorporating hierarchical residual connections to enable the network to focus on learning high-frequency information. Experimental results on a publicly available remote sensing dataset demonstrate the superiority of the proposed method compared to existing approaches.