Efficiently Amalgamated CNN-Transformer Network for Image Super-Resolution Reconstruction

Currently, heavy and sophisticated neural network models are designed to improve image super-resolution reconstruction accuracy. However, the model requires high computation resources and is difficult to deploy on mobile devices. Therefore, designing an efficient lightweight image super-resolution network is urgent. We propose to apply the self-attention mechanism in transformer to compensate for the limitations of convolutional neural networks in global representation. In particular, we apply the split depth-wise transposed orthogonal transformer encoder, which splits the input tensor into feature subsets, and depth-wise separable convolution and self-attention across channel dimensions to implicitly increase the receptive field and encode multi-scale features. Experimental results confirmed that our proposed lightweight image super-resolution network (ACTNet) based on an amalgamate CNN-Transformer can achieve better performance than state-of-the-art methods.