Scholarlink

Where do generative AI (GenAI) tools stand in 2025 when it comes to the design of Interactive Digital Narratives (IDNs)? In 2024, we introduced a first effort in benchmarking GenAI tools for IDN design [14]. This paper updates the findings for the current generation of previously evaluated tools (ChatGPT, Claude, Gemini, Copilot), and also adds several open source entries such as DeepSeek, Llama, Mistral, and Qwen. We introduce two key additions to the GenAI4IDN benchmarking framework: methods to benchmark iterative prompting and the effects of contextualized prompting. Furthermore, we introduce and apply a method for AI-assisted evaluation, facilitating rapid evaluation of large amounts of IDN design artifacts with a level of quality similar to human raters.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

The GenAI4IDN Benchmark 2.0 - AI-Assisted Assessment of Generative AI Tools for the Design of Interactive Digital Narratives

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Hartmut Koenitz,
  • Jonathan Barbara,
  • Mirjam Palosaari Eladhari

摘要

Where do generative AI (GenAI) tools stand in 2025 when it comes to the design of Interactive Digital Narratives (IDNs)? In 2024, we introduced a first effort in benchmarking GenAI tools for IDN design [14]. This paper updates the findings for the current generation of previously evaluated tools (ChatGPT, Claude, Gemini, Copilot), and also adds several open source entries such as DeepSeek, Llama, Mistral, and Qwen. We introduce two key additions to the GenAI4IDN benchmarking framework: methods to benchmark iterative prompting and the effects of contextualized prompting. Furthermore, we introduce and apply a method for AI-assisted evaluation, facilitating rapid evaluation of large amounts of IDN design artifacts with a level of quality similar to human raters.