Unsupervised Pansharpening Using ConvNets

Satellite remote sensing provides detailed, large-scale Earth images. Many applications rely on this information, and there is a strong demand for more and better data. No single sensor provides all the information of interest, which motivates the growing appeal of data fusion. Due to the limitations of the sensors, the acquired images cannot have simultaneously high spatial and spectral resolution. To overcome this problem, two coupled sensors can be used, acquiring a high-resolution panchromatic image and a low-resolution multispectral image. The fusion technique known as pansharpening aims to fuse them to obtain an ideal high-resolution multispectral image. Many model-based pansharpening methods have been developed in recent decades. Recently, research has shifted towards data-driven solutions, hoping to replicate the successes observed in other application fields. The results, however, did not meet these high expectations. This is likely due to the lack of full-resolution real-world data, which prevents the use of supervised learning. Many models are trained on low-resolution synthetic data to circumvent this limitation and then used on the high-resolution data of interest. This approach, however, is based on a dubious assumption of scale invariance and provides questionable results. This chapter proposes a new training framework that works on original high-resolution images, avoiding downscaling and consequent impairments. The framework encompasses novel methods to evaluate the spectral and spatial fidelity of the pansharpened image compared to the original multispectral and panchromatic data. Experiments on real data demonstrate that the proposed methods outperform the current state of the art.