Scholarlink

<p>Benchmarking is essential for evaluating the capabilities of large language models (LLMs). However, existing multimodal benchmarks lack dedicated resources for traditional Chinese opera, a domain rich in cultural and visual complexity. To address this gap, we introduce the TCO-Dataset, a bilingual multimodal dataset designed to assess LLMs’ ability to interpret and reason about Chinese opera images. The dataset contains 1,000 multiple-choice questions paired with high-resolution images across eight major opera genres. Each sample includes a carefully selected image, a corresponding question focused on cultural and visual understanding, and an annotated answer for evaluation. The dataset supports both Chinese and English, enabling cross-lingual model assessment. All items were reviewed through multiple rounds of expert validation to ensure consistency and accuracy. The TCO-Dataset supports diverse applications, including still-image-based visual-cultural reasoning, cultural heritage preservation, and domain-specific AI development. Initial evaluations show significant performance variation across models, underscoring the dataset’s challenge and value for advancing multimodal understanding.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A multimodal benchmark dataset for evaluating large language models on traditional Chinese opera understanding

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月1日

查看全文
  • Gengxian Cao,
  • Shuo Hou,
  • Ye Yang,
  • Ruosong Liu,
  • Hong Duan,
  • Donghe Li

摘要

Benchmarking is essential for evaluating the capabilities of large language models (LLMs). However, existing multimodal benchmarks lack dedicated resources for traditional Chinese opera, a domain rich in cultural and visual complexity. To address this gap, we introduce the TCO-Dataset, a bilingual multimodal dataset designed to assess LLMs’ ability to interpret and reason about Chinese opera images. The dataset contains 1,000 multiple-choice questions paired with high-resolution images across eight major opera genres. Each sample includes a carefully selected image, a corresponding question focused on cultural and visual understanding, and an annotated answer for evaluation. The dataset supports both Chinese and English, enabling cross-lingual model assessment. All items were reviewed through multiple rounds of expert validation to ensure consistency and accuracy. The TCO-Dataset supports diverse applications, including still-image-based visual-cultural reasoning, cultural heritage preservation, and domain-specific AI development. Initial evaluations show significant performance variation across models, underscoring the dataset’s challenge and value for advancing multimodal understanding.