In recent years, substantial advances have been made in deep learning-based image compression. Most studies have focused on designing accurate and flexible entropy models to predict the distribution of latent features in images. However, the allocation of computing resources and the restoration of decoded images by post-processing are equally important. In this paper, we propose asymmetric learned image compression based on fast residual channel attention. We design an asymmetric image compression network to effectively allocate computational resources into the post-processing of the decoder. Inspired by image super-resolution, we provide a fast residual channel attention module in the post-processing based on depthwise separable convolution. This module can quickly restore the features lost by compression, resulting in image quality enhancement. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods for learned image compression in terms of PSNR, MS-SSIM and runtime.