ELOQUENT is a CLEF lab for evaluating generative language model quality with a focus on such aspects of quality that do not come to the fore with current standard test suites and test collections and to develop and promote new test regimes and methods that fit a multilingual application scenario for generative artificial intelligence. This year is the second year of ELOQUENT. This year’s experiment tracks have evolved from the first year: this year we continue challenging the capability of classifiers to distinguish machine-generated from human-authored text; we explore how consistent language models are in responding to value-oriented questions across languages and system settings; we test how accurately language models are able to predict human preferences between variants of generated material; and we investigate how well language models are able provide sensible topical quizzes to fit given target texts.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Overview of ELOQUENT 2025: Shared Tasks for Evaluating Generative Language Model Quality

  • Jussi Karlgren,
  • Ekaterina Artemova,
  • Ondřej Bojar,
  • Marie Isabel Engels,
  • Vladislav Mikhailov,
  • Pavel Šindelář,
  • Erik Velldal,
  • Lilja Øvrelid

摘要

ELOQUENT is a CLEF lab for evaluating generative language model quality with a focus on such aspects of quality that do not come to the fore with current standard test suites and test collections and to develop and promote new test regimes and methods that fit a multilingual application scenario for generative artificial intelligence. This year is the second year of ELOQUENT. This year’s experiment tracks have evolved from the first year: this year we continue challenging the capability of classifiers to distinguish machine-generated from human-authored text; we explore how consistent language models are in responding to value-oriented questions across languages and system settings; we test how accurately language models are able to predict human preferences between variants of generated material; and we investigate how well language models are able provide sensible topical quizzes to fit given target texts.