Scholarlink

In order to systematically assess the ability of large language models (LLMs) to understand and use Chinese language and literature, this study constructs the Chinese Literature Game Arena (CLGA). Centered on a competitive framework of “LLM versus LLM”, the CLGA uses three types of Chinese language games, Idiom Solitaire, Flying Flower Order, and Couplet Pairing, which possess both cultural richness and rules-based attributes, as evaluation scenarios. The study selects 10 LLMs that cover the main technological pathways from domestic and international sources, including the GPT series, the Claude series, the Gemini series, Qwen2.5-72B-Instruct and other benchmark models. A multistage round-robin and elimination tournament system is designed within the CLGA, establishing a quantitative scoring system based on compliance, output quality, response efficiency, and mastery of common versus obscure content. A visualization method is introduced to present the models’ interactive logic. The study reveals the boundaries of the Chinese linguistic competence of these LLMs and provides references for their optimization in cultural domain adaptation.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

CLGA: A Linguistic Arena for Testing LLMs’ Prowess in Chinese Literary Games

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Ziying Wang,
  • Xiahui Ren,
  • Jiahao Wang,
  • Farong Wen,
  • Tao Zhang,
  • Zixuan Luo,
  • Wenxi Sun,
  • Yingjie Zhou,
  • Yu Zhou

摘要

In order to systematically assess the ability of large language models (LLMs) to understand and use Chinese language and literature, this study constructs the Chinese Literature Game Arena (CLGA). Centered on a competitive framework of “LLM versus LLM”, the CLGA uses three types of Chinese language games, Idiom Solitaire, Flying Flower Order, and Couplet Pairing, which possess both cultural richness and rules-based attributes, as evaluation scenarios. The study selects 10 LLMs that cover the main technological pathways from domestic and international sources, including the GPT series, the Claude series, the Gemini series, Qwen2.5-72B-Instruct and other benchmark models. A multistage round-robin and elimination tournament system is designed within the CLGA, establishing a quantitative scoring system based on compliance, output quality, response efficiency, and mastery of common versus obscure content. A visualization method is introduced to present the models’ interactive logic. The study reveals the boundaries of the Chinese linguistic competence of these LLMs and provides references for their optimization in cultural domain adaptation.