Meta-learning with Dual-Branch Fusion Transformer for Hyperspectral Image Super-Resolution
摘要
Hyperspectral image (HSI) super-resolution (SR) aims to enhance the spatial resolution of HSIs, conditioning on high-resolution (HR) multispectral images (MSI). Recently, unsupervised learning-based approaches have shown a growing interest in the field of HSI-SR as they do not rely on the supervision of simulated input-output pairs. However, such methods not only require high-performance computational resources but are also prone to overfit. To address this issue, this paper proposes a meta-learning-based HSI-SR method (MetaHSR) that learns the knowledge of a specific HSI via only one model update iteration. During training, knowledge distillation is utilized as a regularization term to avoid overfitting. Furthermore, a Dual-Branch Fusion Transformer (DBFT) is designed, where local, global, and cross-modal dependencies are learned with convolutions, self-attention, and cross-modal attention, respectively. Extensive experiments demonstrate the superiority of the proposed MetaHSR with respect to state-of-the-art methods.