Scholarlink

<p>Teachers’ use of criterial and individual reference norms offers advantages for formative assessment. Therefore, in this randomized (mixed within-between-person) replication study, we investigate which reference norms teachers apply when interpreting learning progress data, whether the graphical representation has an influence on this and whether the general reference norm orientation can predict the situationally applied reference norm. This conceptual replication study not only offers the opportunity to confirm an effect and its direction already demonstrated in the original study, but we also strengthen the internal, external and construct validity by conducting the study with in-service teachers instead of pre-service teachers and avoiding carry-over effects through design changes. In the study, teachers interpret learning progress data of a&#xa0;school class first in a&#xa0;neutral graphical representation using an open-ended item. The basis for the second interpretation is a&#xa0;graphical representation of the learning progress data of a&#xa0;second class, in which one of the three reference norms (individual, criterial or social) is graphically highlighted. To measure the applied reference norm, we use an open-ended and a&#xa0;single-choice item. Finally, we assess the teachers’ reference norm orientation. In a&#xa0;neutral graphical representation of learning progress data, <i>N</i> = 157 teachers most frequently used one reference norm in their interpretation, namely the individual reference norm. The general reference norm orientation could predict the reference norm used in a&#xa0;specific assessment situation only to a&#xa0;limited extent. In addition, graphical representation showed consistent, conclusive effects of small to large magnitude (2.38 ≤ <i>OR</i> ≤ 7.05) on the situationally applied reference norm.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Effects of graphical representations on teachers’ interpretation of learning progress data. A registered replication

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月20日

查看全文
  • Erika Lunowa,
  • Sarah Bez,
  • Tom Rosman,
  • Samuel Merk

摘要

Teachers’ use of criterial and individual reference norms offers advantages for formative assessment. Therefore, in this randomized (mixed within-between-person) replication study, we investigate which reference norms teachers apply when interpreting learning progress data, whether the graphical representation has an influence on this and whether the general reference norm orientation can predict the situationally applied reference norm. This conceptual replication study not only offers the opportunity to confirm an effect and its direction already demonstrated in the original study, but we also strengthen the internal, external and construct validity by conducting the study with in-service teachers instead of pre-service teachers and avoiding carry-over effects through design changes. In the study, teachers interpret learning progress data of a school class first in a neutral graphical representation using an open-ended item. The basis for the second interpretation is a graphical representation of the learning progress data of a second class, in which one of the three reference norms (individual, criterial or social) is graphically highlighted. To measure the applied reference norm, we use an open-ended and a single-choice item. Finally, we assess the teachers’ reference norm orientation. In a neutral graphical representation of learning progress data, N = 157 teachers most frequently used one reference norm in their interpretation, namely the individual reference norm. The general reference norm orientation could predict the reference norm used in a specific assessment situation only to a limited extent. In addition, graphical representation showed consistent, conclusive effects of small to large magnitude (2.38 ≤ OR ≤ 7.05) on the situationally applied reference norm.