Scholarlink

This study explores the integration of Large Language Models (LLMs), specifically GPT-4o, with machine learning (ML) to automatically detect struggle behaviors in a science exploration game. We evaluate GPT-4o’s ability to analyze text replays, which convert raw gameplay logs into human-readable sequences of player actions, against existing machine-learned struggle detectors. The LLM and ML approaches achieve comparable performance but have complementary strengths. GPT-4o is effective at identifying struggle in instances where the human-engineered features used in traditional ML models fail to capture meaningful behavioral patterns, while ML detectors perform better at identifying struggle in tasks that involve numerical reasoning (e.g., frequent pauses) and in more complex tasks where structured features provide clearer signals.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Integrating Large Language Models and Machine Learning to Detect Struggle in Educational Games

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Xiner Liu,
  • Zhanlan Wei,
  • Ryan S. Baker,
  • Shari J. Metcalf,
  • Jiayi Zhang,
  • Amanda Barany,
  • Stefan Slater,
  • Luke Swanson,
  • David J. Gagnon

摘要

This study explores the integration of Large Language Models (LLMs), specifically GPT-4o, with machine learning (ML) to automatically detect struggle behaviors in a science exploration game. We evaluate GPT-4o’s ability to analyze text replays, which convert raw gameplay logs into human-readable sequences of player actions, against existing machine-learned struggle detectors. The LLM and ML approaches achieve comparable performance but have complementary strengths. GPT-4o is effective at identifying struggle in instances where the human-engineered features used in traditional ML models fail to capture meaningful behavioral patterns, while ML detectors perform better at identifying struggle in tasks that involve numerical reasoning (e.g., frequent pauses) and in more complex tasks where structured features provide clearer signals.