Image inpainting based on fusion structure information and pixelwise attention

Image inpainting refers to restoring the damaged areas of an image using the remaining available information. In recent years, deep learning-based image inpainting has been extensively explored and shown remarkable performance, among which the parallel prior embedding methods have the advantages of few network parameters and relatively low training difficulty. However, most methods use a single prior that is unable to provide sufficient guidance information. Hence, they are unable to generate high-quality, realistic, and vivid images. Fusion labels are effective priors that could provide more meaningful guidance information for inpainting. Meanwhile, attention mechanisms can focus on effective features and establish long-range correlations, which is helpful to refine texture details. Therefore, this paper proposes a parallel prior embedding image inpainting method based on fusion structure information (FSI) and pixelwise attention. A FSI module using the color structure and edge information is designed to update structure features and image features alternately, pixelwise attention is utilized to refine image details, and a joint loss is applied to constrain model training. Extensive experiments are conducted on multiple public datasets, and the results show that the proposed method achieves generally superior performance over several compared methods in terms of several quantitative metrics and qualitative analysis.