Pansharpening is to fuse a high-resolution panchromatic image with a low-resolution hyperspectral image in order to produce a hyperspectral image with enhanced spectral and spatial resolution. However, existing pansharpening algorithms fail to accurately recover spectral information across continuous spectral bands and ranges, and often neglect the preservation of spatial information, leading to spectral distortion and spatial detail loss. To address these issues, a deep learning model based on a two-headed feature soft attention mechanism and a multi-scale convolutional residual block is proposed. Features are first extracted using the multi-scale convolution module, followed by further fusion and enhancement through the depthwise separable convolution residual module. Additionally, a two-headed soft attention mechanism is incorporated to adaptively assign weights to features across both spatial and spectral dimensions, thus enhancing the model’s focus on key regions and important spectral information. To assess the effectiveness of the proposed algorithm, the fused images are assessed from both subjective qualitative and objective quantitative perspectives. Experimental results indicate that the proposed method demonstrates notable improvements in maintaining spatial and spectral information compared with traditional algorithms and recent deep learning methods with respect to the preservation of spatial and spectral information.