Scholarlink

<p>This paper presents a case study of how an undetected wording change in the OECD’s Programme for International Student Assessment (PISA) may have led to erroneous conclusions. We focus on a survey question intended to measure student truancy across countries and over time. While the international documentation suggests that the same question was used across PISA cycles, national documentation reveals that the wording changed in some countries in 2015. We show how this change could easily be missed and how it may affect inferences about trends in truancy, particularly when comparing rates before and after the COVID-19 pandemic. We also examine whether artificial intelligence and large language models could help identify the problem. Instead, these tools provided overconfident but incorrect advice.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

When technical standards fail. Questionnaire wording and the comparability of PISA truancy data

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月31日

查看全文
  • John Jerrim,
  • Maria Palma Carvajal,
  • Jake Anders,
  • María Ladrón de Guevara Rodríguez,
  • Oscar David Marcenaro-Gutierrez

摘要

This paper presents a case study of how an undetected wording change in the OECD’s Programme for International Student Assessment (PISA) may have led to erroneous conclusions. We focus on a survey question intended to measure student truancy across countries and over time. While the international documentation suggests that the same question was used across PISA cycles, national documentation reveals that the wording changed in some countries in 2015. We show how this change could easily be missed and how it may affect inferences about trends in truancy, particularly when comparing rates before and after the COVID-19 pandemic. We also examine whether artificial intelligence and large language models could help identify the problem. Instead, these tools provided overconfident but incorrect advice.