TransDDPM: Transformer-Based Denoising Diffusion Probabilistic Model for Image Restoration

Although diffusion models have achieved impressive success for image generation, its application for image restoration is still underexplored. Following tremendous success in natural language processing, transformers have also shown great success for computer vision. Although several researches indicate that increasing transformer depth/width improves the applicability of diffusion models, application of Transformers in diffusion models is still underexplored due to quadratic complexity with the spatial resolution. In this work, we proposed a Transformer-based Denoising Diffusion Probabilistic Model (TransDDPM) for image restoration. With multi-head cross-covariance attention (MXCA), TransDDPM can operates global self-attention with cross-covariance matrix in channel dimension rather than spatial dimension. Another gated feed-forward network (GFFN) is included to enhance the ability to exploit spatial local context. Powered by these designs, TransDDPM is capable for both long-range dependencies and short-range dependencies and flexible for images of various resolutions. Comprehensive experiments demonstrate our TransDDPM achieves state-of-the art performance on several restoration tasks, e.g., image deraining, image dehazing and motion deblurring.