The paper presents a new way of generating image captions using the pre-trained models, llama-2 and COCO-DetR. The COCO-DetR model was used for object detection while llama-2 was used for captioning the detected objects. To evaluate the system’s accuracy in terms of contextual relevance, and ability to create meaningful captions, the authors compared the outputs of the proposed model with the gold standard COCO dataset. The proposed model demonstrates promising results but fails to address limitations such as language model hallucinations and dependency on the multiple numbers of object detection models.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

VisionCraft: Advanced Image Captioning with Pre-trained Models

  • Vinayaka A. Belludi,
  • Sumeet S. Inamdar,
  • Kaushik Mallibhat,
  • Nalini C. Iyer

摘要

The paper presents a new way of generating image captions using the pre-trained models, llama-2 and COCO-DetR. The COCO-DetR model was used for object detection while llama-2 was used for captioning the detected objects. To evaluate the system’s accuracy in terms of contextual relevance, and ability to create meaningful captions, the authors compared the outputs of the proposed model with the gold standard COCO dataset. The proposed model demonstrates promising results but fails to address limitations such as language model hallucinations and dependency on the multiple numbers of object detection models.