Multi-objective Evolution of Diffusion Model Prompt Embeddings Using CLIP-IQA
摘要
We study multi- and many-objective evolutionary optimization of image quality criteria in diffusion models, with the goal of characterizing the capabilities and limits of a diffusion model and an image quality scorer. Our search operates exclusively in the prompt embedding space: we evolve continuous embeddings that condition image generation, without using or modifying textual prompts. This representation is well suited for real valued evolutionary algorithms, enables fine-grained control, and avoids constraints imposed by discrete word-level prompt manipulation. Multi-objective optimization is employed as an analysis tool to examine which score ranges are achievable, how performance scales with an increasing number of objectives, and where conflicts between image quality criteria arise under a fixed generator and evaluator. Prompt embeddings are evolved from random initialization while keeping model parameters unchanged. Image quality is assessed using CLIP-IQA with native and custom axes such as “naturalness”, “complexity”, and “happiness”. Experiments with three, four, five, and ten objectives compare a single-objective Genetic Algorithm based on summed scores with NSGA-III. Results show that embedding evolution consistently improves scores over random initialization and reveals trade-offs between visual criteria, with NSGA-III achieving higher hypervolume and greater diversity than the single-objective baseline.