Scholarlink

<p>The objective of pansharpening and hypersharpening is to accurately fuse a high-resolution panchromatic (PAN) image with a low-resolution multispectral (MS) or hyperspectral (HS) image, respectively. Unfolding fusion methods integrate the powerful representation capabilities of deep learning with the robustness of model-based approaches. These techniques usually involve unrolling the steps of the optimization scheme derived from the minimization of a variational energy into a deep learning framework, resulting in efficient and highly interpretable architectures. In this paper, we present a model-based deep unfolded method for satellite image fusion. Our approach relies on a variational formulation that incorporates the classic observation model for MS/HS data, a high-frequency injection constraint, and a general prior. For the unfolding stage, we design upsampling and downsampling layers that leverage geometric information encoded in the PAN image through residual networks. The core of our method is a Multi-Head Attention Residual Network (MARNet), which combines multiple head attentions with residual learning to capture image self-similarities using nonlocal patch-based operators. Additionally, we include a post-processing module based on the MARNet architecture to further enhance the quality of the fused images. Experimental results on PRISMA, QuickBird, and WorldView2 datasets demonstrate the superior performance of our method, both at reduced and full-scale resolutions, along with its ability to generalize across different sensor configurations and varying spatial and spectral resolutions. The source code will be available at <a href="https://github.com/TAMI-UIB/MARNet">https://github.com/TAMI-UIB/MARNet</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multi-Head Attention Residual Unfolded Network for Model-Based Pansharpening

  • 期刊论文

  • 发表日期：

    2026年1月12日

查看全文
  • Ivan Pereira-Sánchez,
  • Eloi Sans,
  • Julia Navarro,
  • Joan Duran

摘要

The objective of pansharpening and hypersharpening is to accurately fuse a high-resolution panchromatic (PAN) image with a low-resolution multispectral (MS) or hyperspectral (HS) image, respectively. Unfolding fusion methods integrate the powerful representation capabilities of deep learning with the robustness of model-based approaches. These techniques usually involve unrolling the steps of the optimization scheme derived from the minimization of a variational energy into a deep learning framework, resulting in efficient and highly interpretable architectures. In this paper, we present a model-based deep unfolded method for satellite image fusion. Our approach relies on a variational formulation that incorporates the classic observation model for MS/HS data, a high-frequency injection constraint, and a general prior. For the unfolding stage, we design upsampling and downsampling layers that leverage geometric information encoded in the PAN image through residual networks. The core of our method is a Multi-Head Attention Residual Network (MARNet), which combines multiple head attentions with residual learning to capture image self-similarities using nonlocal patch-based operators. Additionally, we include a post-processing module based on the MARNet architecture to further enhance the quality of the fused images. Experimental results on PRISMA, QuickBird, and WorldView2 datasets demonstrate the superior performance of our method, both at reduced and full-scale resolutions, along with its ability to generalize across different sensor configurations and varying spatial and spectral resolutions. The source code will be available at https://github.com/TAMI-UIB/MARNet.