Hyperspectral image (HSI) reconstruction seeks to recover three-dimensional spatial-spectral data from two-dimensional measurements acquired with the coded aperture snapshot spectral imaging (CASSI) system. Recently, the deep unfolding framework has shown significant advancements in HSI reconstruction through its use of linear projection and denoising components. Nonetheless, current approaches continue to encounter various challenges. These issues can be summarized into two aspects: 1) They do not consider the similarity of features between stages, resulting in inadequate handling of feature similarity and limiting the utilization of information from previous stages; 2) They are unable to simultaneously and comprehensively predict the noise level, degradation patterns, and the degree of ill-posedness to effectively guide iterative learning. To tackle these challenges, this paper introduces a Dual-Cross Fusion Deep-unfolding Transformer (DCFDT) for the HSI reconstruction task. First, our proposed Dual-Cross Fusion Attention Block (D-CFAB) captures more detailed spatial information and models feature similarity between adjacent stages by applying cross-attention between features of the previous and current stages. Second, we consider the measured values and the physical mask while comprehensively predicting the current stage’s noise level using information from the previous stage. Additionally, in the linear projection module, we utilize channel attention to capture the degradation effects and challenges introduced by mask modulation and dispersion integration, ensuring thorough guidance for each iteration. Experimental findings, obtained from both simulated and actual scenarios, highlight the enhanced efficacy of our approach relative to advanced HSI reconstruction methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dual Cross Fusion Deep-Unfolding Transformer for Hyperspectral Image Reconstruction

  • Chao Wang,
  • Shuai Gao,
  • Xinming Sun,
  • Shiji Liu,
  • Wanli Lv

摘要

Hyperspectral image (HSI) reconstruction seeks to recover three-dimensional spatial-spectral data from two-dimensional measurements acquired with the coded aperture snapshot spectral imaging (CASSI) system. Recently, the deep unfolding framework has shown significant advancements in HSI reconstruction through its use of linear projection and denoising components. Nonetheless, current approaches continue to encounter various challenges. These issues can be summarized into two aspects: 1) They do not consider the similarity of features between stages, resulting in inadequate handling of feature similarity and limiting the utilization of information from previous stages; 2) They are unable to simultaneously and comprehensively predict the noise level, degradation patterns, and the degree of ill-posedness to effectively guide iterative learning. To tackle these challenges, this paper introduces a Dual-Cross Fusion Deep-unfolding Transformer (DCFDT) for the HSI reconstruction task. First, our proposed Dual-Cross Fusion Attention Block (D-CFAB) captures more detailed spatial information and models feature similarity between adjacent stages by applying cross-attention between features of the previous and current stages. Second, we consider the measured values and the physical mask while comprehensively predicting the current stage’s noise level using information from the previous stage. Additionally, in the linear projection module, we utilize channel attention to capture the degradation effects and challenges introduced by mask modulation and dispersion integration, ensuring thorough guidance for each iteration. Experimental findings, obtained from both simulated and actual scenarios, highlight the enhanced efficacy of our approach relative to advanced HSI reconstruction methods.