Text2Printing: Controllable Textile Digital Printing Pattern Generation with Attention Modulation
摘要
Recent text-to-image diffusion models have witnessed impressive capabilities in generating diverse and creative images from textual prompts. However, most existing text-to-image generation approaches tend to inaccurate layout control and multi-concept semantic deviations when directly applied to generate textile digital printing patterns under the guidance of given layout conditions. To address the above issues, we propose Text2Printing, a novel controllable text-to-image generation framework based on attention modulation. Specifically, we fine-tune the ControlNet model with bounding box constraints and employs attention modulation to increase attention weights for target objects within their designated boxes, thereby guaranteeing the generated patterns within the expected bounding regions. To better preserve the compositional semantics described in the text prompts, we develop a linguistic structure guidance strategy to reconstruct semantic mapping relationships within the cross-attention layers. Furthermore, we introduce FreeU scheme to refine the U-Net backbone and high-frequency feature weights during the diffusion process, resulting in enhanced image quality and fidelity. Comprehensive experiments demonstrate that our method obviously outperforms state-of-the-art layout control approaches in achieving precise spatial control and generating semantically consistent target objects.