<p>Lightweight image super-resolution (SR) requires effective deployment on edge devices under strict computational cost constraints while accurately modeling long-range dependencies. Convolutional neural networks (CNNs) offer low-latency characteristics but struggle to capture non-local features, while Transformers excel at modeling long-range dependencies but suffer from slow inference speed due to the quadratic complexity of Softmax attention. Linear attention serves as an efficient compromise; however, vanilla linear attention maps different queries to identical attention weights due to the non-injective nature of ReLU, resulting in a lack of spatial feature perception capability, making it challenging for direct application in image super-resolution, limiting performance. To alleviate this limitation, we propose Perception-enhanced Linear Attention (PeLA) specifically designed for lightweight image super-resolution. By integrating positional encoding and a feature perception mechanism, PeLA enhances the weights of critical features during attention computation. This enables linear attention to obtain distinct attention weights in a manner analogous to Softmax attention, allowing it to focus on more salient information while introducing only minimal computational overhead. Furthermore, to optimize features in conjunction with PeLA, we introduce a Refined Feature Feed-forward Network (RFFN), which refines effective features through partial channel refinement and dynamic gating, suppressing redundant features. Based on these two core components, we develop PeLASR, a lightweight SR framework. Extensive experiments demonstrate that the proposed PeLASR family outperforms existing state-of-the-art lightweight image SR methods based on CNN, Transformer, and Mamba, while maintaining low computational cost. The code is available at <a href="https://github.com/sxdyyds/PeLASR">https://github.com/sxdyyds/PeLASR</a>.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

PeLA: perception-enhanced linear attention for lightweight image super-resolution

  • Yizhi Cong,
  • Baoting Wang,
  • Hongyan Guo,
  • Kai Wang

摘要

Lightweight image super-resolution (SR) requires effective deployment on edge devices under strict computational cost constraints while accurately modeling long-range dependencies. Convolutional neural networks (CNNs) offer low-latency characteristics but struggle to capture non-local features, while Transformers excel at modeling long-range dependencies but suffer from slow inference speed due to the quadratic complexity of Softmax attention. Linear attention serves as an efficient compromise; however, vanilla linear attention maps different queries to identical attention weights due to the non-injective nature of ReLU, resulting in a lack of spatial feature perception capability, making it challenging for direct application in image super-resolution, limiting performance. To alleviate this limitation, we propose Perception-enhanced Linear Attention (PeLA) specifically designed for lightweight image super-resolution. By integrating positional encoding and a feature perception mechanism, PeLA enhances the weights of critical features during attention computation. This enables linear attention to obtain distinct attention weights in a manner analogous to Softmax attention, allowing it to focus on more salient information while introducing only minimal computational overhead. Furthermore, to optimize features in conjunction with PeLA, we introduce a Refined Feature Feed-forward Network (RFFN), which refines effective features through partial channel refinement and dynamic gating, suppressing redundant features. Based on these two core components, we develop PeLASR, a lightweight SR framework. Extensive experiments demonstrate that the proposed PeLASR family outperforms existing state-of-the-art lightweight image SR methods based on CNN, Transformer, and Mamba, while maintaining low computational cost. The code is available at https://github.com/sxdyyds/PeLASR.