Dual-dimension modulation aggregation network for lightweight image super-resolution
摘要
Transformer-based restoration methods achieve significant performance as the self-attention (SA) of the Transformer can explore non-local information for better high-resolution image reconstruction. However, their computational complexity scales quadratically with increasing spatial resolution. Existing solutions reduce attention cost through local windows or category priors, but often introduce extra computation or pretrained dictionaries. Furthermore, for images with repeated structures, edges, and high-frequency textures, the reconstruction of a local region often depends more on several spatially correlated regions than on all input pixels uniformly. To address these limitations and leverage the aforementioned observation, we propose a lightweight dual-dimension modulation aggregation network, which combines channel-wise and spatial feature interactions to achieve more accurate reconstruction. Specifically, we employ a Self-Modulation Feature Aggregation (SMFA) block to capture long-range dependencies via variance modulation within the channel dimension. Crucially, a Cascaded Regional Self-Attention (CRSA) block is designed to refine key pixels in the spatial dimension. By leveraging a dual-cascading strategy with progressively increasing receptive fields, CRSA effectively aggregates multi-scale local details and models long-range dependencies. Extensive experiments on public benchmarks show that DMANet achieves competitive reconstruction performance with lower model complexity and runtime overhead.