Enhancing Image-to-Image Translation with Contrast Loss Constrained Generators and Selective Neighborhood Sampling

Unpaired image-to-image (I2I) translation has been a popular research topic in the field of computer vision, and researchers have recently obtained the desired images by introducing contrastive loss to constrain the generative process of generators. However, when constructing negative samples for contrastive learning, they use a random sampling strategy, which may affect the performance of the model. In this paper, we address this issue by deliberately selecting image blocks with high overlapping semantic information. We designed an SNS module to compute a similarity matrix by comparing the feature distances between the original image blocks and the generated images, each row of this matrix corresponds to probability distribution between the image patches, and we select the few positions with the highest probability for sampling. In addition, we note that the domain invariant information in the generated images is subject to problems such as distortion and discoloration, and we use the attention mechanism to separate the domain invariant information and use this information as the final result of the auxiliary computation. We validate our proposed method on three different image translation datasets, showing that it improves the quality of the generated images without adding additional learnable parameters.