Scholarlink

In this paper, we introduce AutoTagGen++, a novel semantic image tagging model that leverages the strengths of large language models (LLMs) and deep learning to achieve superior image tagging performance. The approach incorporates the Gemini model and an image captioning pipeline. The pipeline employs a Convolutional Neural Network (CNN) - Long Short-Term Memory (LSTM) network to generate captions for images. To extract tags from these generated captions, a probabilistic parser is used, focusing on objects, scenes, attributes, and actions leading to more relevant tags. Furthermore, AutoTagGen++ leverages Gemini’s zero-shot learning capabilities to enrich the tag set with additional concepts. External knowledge graphs are also incorporated to further refine the tags by identifying relevant entities and relationships. Finally, a semantic filtering step ensures a high-quality enriched final tag set. Comprehensive assessments across various benchmarks including NUS-WIDE, MS-COCO, Pascal VOC 2012, OpenImages, and MIR-Flickr25K indicate that AutoTagGen++ outperforms existing state-of-the-art (SOTA) image tagging models in terms of recall, precision, Mean Average Precision (mAP) score and F1-score.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

AutoTagGen: A Semantic Approach for Image Tagging Utilising Large Language Models and Community Verified Integrative Knowledge

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Swati Sampada Parida,
  • Raghav Khullar,
  • Manav Aggarwal,
  • Gerard Deepak,
  • A. Santhanavijayan

摘要

In this paper, we introduce AutoTagGen++, a novel semantic image tagging model that leverages the strengths of large language models (LLMs) and deep learning to achieve superior image tagging performance. The approach incorporates the Gemini model and an image captioning pipeline. The pipeline employs a Convolutional Neural Network (CNN) - Long Short-Term Memory (LSTM) network to generate captions for images. To extract tags from these generated captions, a probabilistic parser is used, focusing on objects, scenes, attributes, and actions leading to more relevant tags. Furthermore, AutoTagGen++ leverages Gemini’s zero-shot learning capabilities to enrich the tag set with additional concepts. External knowledge graphs are also incorporated to further refine the tags by identifying relevant entities and relationships. Finally, a semantic filtering step ensures a high-quality enriched final tag set. Comprehensive assessments across various benchmarks including NUS-WIDE, MS-COCO, Pascal VOC 2012, OpenImages, and MIR-Flickr25K indicate that AutoTagGen++ outperforms existing state-of-the-art (SOTA) image tagging models in terms of recall, precision, Mean Average Precision (mAP) score and F1-score.