Image Style Transfer Enhancement Through Enriched Text
摘要
Style transfer methodologies aim to morph the content of an image by adopting the semantic textures from a stylistic reference image. However, these methods often face practical challenges due to the unavailability of suitable style reference images. To address this, text-driven image stylization has been introduced. Nonetheless, these techniques encounter difficulties stemming from the discrepancies between textual descriptions and visual representations. Specifically, Traditional models that are trained solely on simple text-image pairs tend to achieve only basic style transformations, which heavily rely on the specific vocabulary used. To overcome these limitations, we propose GPT-CLIPstyler, a novel text-driven model enhanced for image style transfer. Our method utilizes an image-to-text model to generate textual descriptions that fully encapsulate the semantic essence of the original content images. By incorporating GPT-4, the model enhances the semantic coherence. It merges the text derived from the content image with the style-defining text, facilitating efficient integration of cross-modal style information. This merged text serves as a dynamic guide throughout the stylization process. Additionally, we introduce a text encoder designed to handle extensive sentence-based style descriptions, which enables more precise image stylizations reflecting complex semantic queries. Extensive experimental evaluations demonstrate that our method not only preserves the structural integrity of the original image but also accurately reflects the complex semantics conveyed in detailed textual descriptions. Comparative analyses with state-of-the-art methods confirm that GPT-CLIPstyler delivers superior stylization outcomes.