Reconstructing Hyperspectral Images (HSIs) from Coded Aperture Snapshot Spectral Imaging (CASSI) is an important yet challenging task. The core issue lies in recovering reliable and detailed 3D HSI cube from 2D measurement. Deep unfolding framework which alternates between solving data subproblems and prior subproblems has made satisfactory progress in HSIs reconstruction task. However, current methods do not fully utilize the spatial spectral prior of HSIs. To solve this problem and further enhance the spectral-spatial representation capabilities in the prior subproblems, we propose a Spatial-Spectral Correlation Transformer Based on Deep Unfolding Framework (SSCDUF). Specifically, we introduce a multi-scale Spatial-Spectral Correlation Fusion Transformer (SSCT) module that simultaneously utilize the similarity and correlation of spectral features as well as local and non-local spatial features, jointly using spatial and spectral prior to enhance feature representation. Moreover, we further propose an Adaptive Aggregation Skip Connection (AASC) module to adaptively aggregate spatial and spectral features in multiple scales. Extensive experimental results on both simulated and real scenes demonstrate that SSCDUF outperforms the state-of-the-art methods in terms of quantitative metrics while maintaining low parameter costs and runtime.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SSCDUF: Spatial-Spectral Correlation Transformer Based on Deep Unfolding Framework for Hyperspectral Image Reconstruction

  • Hui Zhao,
  • Na Qi,
  • Qing Zhu,
  • Xiumin Lin

摘要

Reconstructing Hyperspectral Images (HSIs) from Coded Aperture Snapshot Spectral Imaging (CASSI) is an important yet challenging task. The core issue lies in recovering reliable and detailed 3D HSI cube from 2D measurement. Deep unfolding framework which alternates between solving data subproblems and prior subproblems has made satisfactory progress in HSIs reconstruction task. However, current methods do not fully utilize the spatial spectral prior of HSIs. To solve this problem and further enhance the spectral-spatial representation capabilities in the prior subproblems, we propose a Spatial-Spectral Correlation Transformer Based on Deep Unfolding Framework (SSCDUF). Specifically, we introduce a multi-scale Spatial-Spectral Correlation Fusion Transformer (SSCT) module that simultaneously utilize the similarity and correlation of spectral features as well as local and non-local spatial features, jointly using spatial and spectral prior to enhance feature representation. Moreover, we further propose an Adaptive Aggregation Skip Connection (AASC) module to adaptively aggregate spatial and spectral features in multiple scales. Extensive experimental results on both simulated and real scenes demonstrate that SSCDUF outperforms the state-of-the-art methods in terms of quantitative metrics while maintaining low parameter costs and runtime.