PROMPTIST: Automated Prompt Optimization for Text-to-Image Synthesis
摘要
Recent advancements in text-to-image synthesis (TIS) models, such as DALL-E 3 and stable diffusion XL, have significantly improved image quality. However, these models typically require experts with artistic and photographic expertise to create model-preferred prompts for high-quality image generation, a requirement that challenges non-experts in real-world applications. This study proposes PROMPTIST, an automated framework that optimizes user-original prompts into model-preferred prompts, thereby enabling TIS models to produce more aesthetically pleasing images. We employ the retrieval augmented generation (RAG) technique with Large Language Models (LLMs) to automatically refine prompts, ensuring that the proposed framework is broadly applicable and extends beyond the specific models’ styles like stable diffusion or DALL-E. Furthermore, to ensure the LLMs generate visually appealing images, we fine-tuned a quality assessment model to analyze prompt quality and provide necessary feedback to the LLMs. The proposed prompt engineering method for LLMs can significantly enhance the quality of both prompts and images. Extensive experimental results consistently demonstrate PROMPTIST’s superiority over strong baselines.