Visual language models have accelerated many multimodal tasks, such as generating image description and answering questions from images or videos. Most work in this field has focused on the English language, in contrast to the few efforts that have addressed Arabic. This paper aims to propose a new model for answering real-world Arabic image questions using the PEFT method on the BLIP2 model and the Arabic version of the DAQUAR dataset (Ara-DAQUAR). As a multi-label classification task, our proposed model yields high performance with a training loss of 0.3916, a validation loss of 0.4408, and a hamming accuracy of 92% after four epochs.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Parameter-Efficient Fine-Tuning of BLIP2 for Arabic Question-Answering on Real-World Images

  • Mohamed Khenchouch,
  • Walid Ounachad,
  • Mohamed Boudchiche,
  • Imad Zeroual,
  • Anoual El kah

摘要

Visual language models have accelerated many multimodal tasks, such as generating image description and answering questions from images or videos. Most work in this field has focused on the English language, in contrast to the few efforts that have addressed Arabic. This paper aims to propose a new model for answering real-world Arabic image questions using the PEFT method on the BLIP2 model and the Arabic version of the DAQUAR dataset (Ara-DAQUAR). As a multi-label classification task, our proposed model yields high performance with a training loss of 0.3916, a validation loss of 0.4408, and a hamming accuracy of 92% after four epochs.