Scholarlink

<p>AI-generated images continue to pose challenges such as misalignment between the text and generated output image and insufficient naturalness in terms of visual quality. These challenges necessitate the need for image quality assessment of the generated image to quantify the alignment of the images with the input text prompts. In this context, the BERT (Bidirectional Encoder Representation from Transformers) model outperforms OpenAI’s GPT on the GLUE (General Language Understanding Evaluation) task. Its expanded tokenization length of upto 512 also helps circumvent CLIP’s limitation of reduced text prompt. Therefore, in the present work, we propose a fusion model that applies BERT as a text encoder combined with CLIP as an image encoder. A bidirectional prompt learning approach through BERT is employed to extract the text features of the prompt used for the generation of the images. Further, using cross attention feature fusion, the proposed method obtains better SRCC and PLCC correlation metric results when compared with state-of-the-art methods on both PKUI2IQA and AGIQA-3K. Results of the ablation study and comparative characterization with other quality assessment metrics for AI-generated images demonstrate a noteworthy performance of the proposed method.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multimodal Quality Assessment of AI-Generated Images using BERT-CLIP Feature Fusion

  • 期刊论文

  • 发表日期：

    2025年10月3日

查看全文
  • P Shabari Nath,
  • Rajlaxmi Chouhan

摘要

AI-generated images continue to pose challenges such as misalignment between the text and generated output image and insufficient naturalness in terms of visual quality. These challenges necessitate the need for image quality assessment of the generated image to quantify the alignment of the images with the input text prompts. In this context, the BERT (Bidirectional Encoder Representation from Transformers) model outperforms OpenAI’s GPT on the GLUE (General Language Understanding Evaluation) task. Its expanded tokenization length of upto 512 also helps circumvent CLIP’s limitation of reduced text prompt. Therefore, in the present work, we propose a fusion model that applies BERT as a text encoder combined with CLIP as an image encoder. A bidirectional prompt learning approach through BERT is employed to extract the text features of the prompt used for the generation of the images. Further, using cross attention feature fusion, the proposed method obtains better SRCC and PLCC correlation metric results when compared with state-of-the-art methods on both PKUI2IQA and AGIQA-3K. Results of the ablation study and comparative characterization with other quality assessment metrics for AI-generated images demonstrate a noteworthy performance of the proposed method.