Image style transfer with saliency constrained and SIFT feature fusion

This article develops a novel image style transfer method that transforms input images using a neural network (NN) model. Common neural style transfer techniques often struggle to fully transmit the texture and color from the style image to the target image (content image), or they may introduce some visible errors. To mitigate these issues, this article proposes a new significance constraint method. Initially, existing significance detection methods are evaluated to select the most appropriate one for our approach. The selected saliency map feature is utilized to detect objects in the style image that correspond to objects with the same saliency map feature in the content image. Furthermore, to address the challenges posed by different sizes or resolutions of style and content images, scale-invariant feature transformations are employed to generate a variety of attribute images. These images are then used to create more feature maps which can be used for patch matching. Consequently, a novel loss function is proposed by associated saliency feature loss, style loss, and content loss. This function also incorporates the gradient of saliency feature constraints into style transfer iterations. At last, the input images and saliency map feature results are employed as multi-channel inputs for the improved deep convolutional neural network (CNN) model for style transfer. Many experimental results demonstrate that the saliency feature map of the source image aids in finding the correct match and avoiding artifacts. Tests on different types of images also show that the proposed method generate better results than other representative methods recently published and deliver superior performance.