Scholarlink

<p>Large Vision-Language Models are an extension of Large Language Models to process signals from the vision-language domain. Their performance is remarkable since they exploit the cutting-edge capabilities of pre-trained Large Language Models. However, most works using these models focus on English language data for training and evaluation. In light of this, several works have been released to extend current training mixtures to consider non-English data. As a matter of fact, several works also considered cultural aspects to decrease bias related to American and Chinese culture. However, evaluation in non-English languages is still limited w.r.t. English. Specifically, there are no datasets that focus on text-centric non-English images, since most available multimodal benchmarks focus on natural images. This limitation makes it unclear whether Large Vision-Language Models can be used for non-English text-centric tasks. In this work, we aim to understand the capabilities of these models for the Italian language and introduce a benchmark dataset focusing on text-centric Italian images. Therefore, we introduce our benchmark dataset <span>ETCII</span> (<i>Evaluation of Text-Centric Italian Images</i>), to evaluate Large Vision-Language Models on three diverse task categories for text-centric images.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Do you understand Italian? Evaluating LVLMs on Italian visual question-answering

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月26日

查看全文
  • Elio Musacchio,
  • Lucia Siciliani,
  • Pierpaolo Basile,
  • Giovanni Semeraro

摘要

Large Vision-Language Models are an extension of Large Language Models to process signals from the vision-language domain. Their performance is remarkable since they exploit the cutting-edge capabilities of pre-trained Large Language Models. However, most works using these models focus on English language data for training and evaluation. In light of this, several works have been released to extend current training mixtures to consider non-English data. As a matter of fact, several works also considered cultural aspects to decrease bias related to American and Chinese culture. However, evaluation in non-English languages is still limited w.r.t. English. Specifically, there are no datasets that focus on text-centric non-English images, since most available multimodal benchmarks focus on natural images. This limitation makes it unclear whether Large Vision-Language Models can be used for non-English text-centric tasks. In this work, we aim to understand the capabilities of these models for the Italian language and introduce a benchmark dataset focusing on text-centric Italian images. Therefore, we introduce our benchmark dataset ETCII (Evaluation of Text-Centric Italian Images), to evaluate Large Vision-Language Models on three diverse task categories for text-centric images.