Scholarlink

<p>Although evaluation benchmarks for general multimodal large language models (MLLMs) are increasingly prevalent, the systematic evaluation of their capabilities for processing ancient texts remains underdeveloped. Ancient books, as cultural heritage artifacts, integrate rich textual and visual elements. Due to their unique cross-linguistic complexity and multimodal composition, they pose challenges to the evaluation of the multifaceted capabilities of MLLMs. To address this issue, we propose benchmarking the ancient book capabilities of MLLMs (BABMLLM), a specialized benchmark designed to evaluate their performance specifically within the domain of ancient books. This benchmark comprises seven curated datasets, enabling comprehensive evaluation across four core tasks relevant to ancient book processing: ancient book translation, text recognition, image captioning, and image-text consistency judgment. Furthermore, BABMLLM provides a standardized reference for evaluating MLLMs in the context of ancient books and establishes a foundation for selecting suitable base models for subsequent domain-specific development.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Benchmarking the ancient books capability of multimodal large language models

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年7月12日

查看全文
  • Dongbo Wang,
  • Dongmei Zhu

摘要

Although evaluation benchmarks for general multimodal large language models (MLLMs) are increasingly prevalent, the systematic evaluation of their capabilities for processing ancient texts remains underdeveloped. Ancient books, as cultural heritage artifacts, integrate rich textual and visual elements. Due to their unique cross-linguistic complexity and multimodal composition, they pose challenges to the evaluation of the multifaceted capabilities of MLLMs. To address this issue, we propose benchmarking the ancient book capabilities of MLLMs (BABMLLM), a specialized benchmark designed to evaluate their performance specifically within the domain of ancient books. This benchmark comprises seven curated datasets, enabling comprehensive evaluation across four core tasks relevant to ancient book processing: ancient book translation, text recognition, image captioning, and image-text consistency judgment. Furthermore, BABMLLM provides a standardized reference for evaluating MLLMs in the context of ancient books and establishes a foundation for selecting suitable base models for subsequent domain-specific development.