This work evaluates three text-to-image generative models: DALL-E 2, Stable Diffusion, and Pix-Art- \(\alpha \) . Initially, the selected models are delineated and three key hyperparameters—prompt, width, and height—are identified for assessment. The prompt parameter is based on a proposed benchmark, while the width and height dimensions are fixed at (512, 512) and (1024, 1024). Subsequently, a multi-task benchmark comprising six task types, each with 12 prompts, is established. Additionally, two methodologies for model evaluation are introduced: human evaluation and the CLIP score metric. This dual approach allows for a comprehensive qualitative and quantitative assessment of the models. The performance of DALL-E 2, Stable Diffusion, and Pix-Art- \(\alpha \) is investigated in relation to the defined hyperparameters, aiming to identify the model that demonstrates superior performance across the benchmark while analyzing the impact of resolution (width and height) on outcomes. The research question posits that substantial performance disparities are anticipated due to the notable variations in the underlying deep learning architectures and training datasets utilized, warranting thorough exploration. The evaluation indicates that the commercial model DALL-E 2 outperforms the free models in both qualitative and quantitative assessments. Notably, the analysis reveals a general deficiency in text-related tasks. Additionally, human features, such as hands, fingers, and pupils, are often inadequately represented in the generated images.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluation of Three Text-to-Image Gen AI Models

  • Evgenios N. Mazarakis

摘要

This work evaluates three text-to-image generative models: DALL-E 2, Stable Diffusion, and Pix-Art- \(\alpha \) . Initially, the selected models are delineated and three key hyperparameters—prompt, width, and height—are identified for assessment. The prompt parameter is based on a proposed benchmark, while the width and height dimensions are fixed at (512, 512) and (1024, 1024). Subsequently, a multi-task benchmark comprising six task types, each with 12 prompts, is established. Additionally, two methodologies for model evaluation are introduced: human evaluation and the CLIP score metric. This dual approach allows for a comprehensive qualitative and quantitative assessment of the models. The performance of DALL-E 2, Stable Diffusion, and Pix-Art- \(\alpha \) is investigated in relation to the defined hyperparameters, aiming to identify the model that demonstrates superior performance across the benchmark while analyzing the impact of resolution (width and height) on outcomes. The research question posits that substantial performance disparities are anticipated due to the notable variations in the underlying deep learning architectures and training datasets utilized, warranting thorough exploration. The evaluation indicates that the commercial model DALL-E 2 outperforms the free models in both qualitative and quantitative assessments. Notably, the analysis reveals a general deficiency in text-related tasks. Additionally, human features, such as hands, fingers, and pupils, are often inadequately represented in the generated images.