Scholarlink

As Large Language Models (LLMs) become increasingly available to students, BPM educators face a new challenge: how to design assignments that remain pedagogically effective and resistant to superficial AI-generated answers. This paper presents the results of two experiments that simulate common BPMN-related homework tasks and test how LLMs respond to them. The first experiment focused on answering comprehension questions based on five BPMN models, each provided either in PNG or XML format. The second asked the models to detect modeling errors in 30 flawed BPMN diagrams. In both cases, we evaluated the outputs of ChatGPT-4o and Gemini Flash, analyzing the correctness, reasoning, and completeness of their responses. Our findings show that while current LLMs are not yet fully capable of reliably solving BPMN assignments—especially those involving deeper process logic—they can already provide partially correct and plausible responses. This raises questions about the future of BPM education, the design of AI-aware assignments, and the role of LLMs as potential learning assistants. Alongside insights and lessons learned, we provide materials to help instructors adapt their teaching to an AI-enabled environment.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Revisiting BPMN Assignments with AI in Mind: Insights from Experiments with Large Language Models in Process Modeling Education

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Krzysztof Kluza,
  • Marzena Grzesiak,
  • Piotr Sliż,
  • Leszek Szała,
  • Anna Suchenia,
  • Karol Suchenia,
  • Marek Moszyński

摘要

As Large Language Models (LLMs) become increasingly available to students, BPM educators face a new challenge: how to design assignments that remain pedagogically effective and resistant to superficial AI-generated answers. This paper presents the results of two experiments that simulate common BPMN-related homework tasks and test how LLMs respond to them. The first experiment focused on answering comprehension questions based on five BPMN models, each provided either in PNG or XML format. The second asked the models to detect modeling errors in 30 flawed BPMN diagrams. In both cases, we evaluated the outputs of ChatGPT-4o and Gemini Flash, analyzing the correctness, reasoning, and completeness of their responses. Our findings show that while current LLMs are not yet fully capable of reliably solving BPMN assignments—especially those involving deeper process logic—they can already provide partially correct and plausible responses. This raises questions about the future of BPM education, the design of AI-aware assignments, and the role of LLMs as potential learning assistants. Alongside insights and lessons learned, we provide materials to help instructors adapt their teaching to an AI-enabled environment.