Object detection in images using natural language prompts has recently become popular due to the extensive development of open-vocabulary perception tasks. We propose an enhancement strategy for open-vocabulary object detection models through a fine-tuning approach named Reframing. The cornerstone of our method is transforming a user’s query into a prompt optimally aligned with the requirements of a specific detection model. This transformation is facilitated by integrating an additional Large Language Model, which undergoes further training. We propose online and offline modes of Reframing, employing deep learning techniques and feedback from the detection models to refine its query transformation process. Our methodology has been applied to various detection models, showcasing its capability to significantly improve prediction accuracy. Developed code and dataset are available at https://github.com/ZoyaV/reframing .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Reframing: Detector-Specific Prompt Tuning for Enhancing Open-Vocabulary Object Detection

  • Mikhail Avshalumov,
  • Zoya Volovikova,
  • Dmitry Yudin,
  • Alexandr Panov

摘要

Object detection in images using natural language prompts has recently become popular due to the extensive development of open-vocabulary perception tasks. We propose an enhancement strategy for open-vocabulary object detection models through a fine-tuning approach named Reframing. The cornerstone of our method is transforming a user’s query into a prompt optimally aligned with the requirements of a specific detection model. This transformation is facilitated by integrating an additional Large Language Model, which undergoes further training. We propose online and offline modes of Reframing, employing deep learning techniques and feedback from the detection models to refine its query transformation process. Our methodology has been applied to various detection models, showcasing its capability to significantly improve prediction accuracy. Developed code and dataset are available at https://github.com/ZoyaV/reframing .