Scholarlink

<p>Situational Judgment Tests (SJTs) are important tools for evaluating individual abilities by presenting specific scenarios and assessing the responses. This study explores the potential of Large Language Models (LLMs) in developing SJTs, with emotional regulation as an example. We proposed and evaluated two approaches using ERNIE 4.0, an advanced Chinese LLM: generating new items from scratch (Approach 1) and adapting items from existing examples (Approach 2). The quality of generated items was assessed through expert evaluations and empirical testing with 93 psychology undergraduates and 184 participants from a general population pool. Results indicate that items generated by ERNIE 4.0, particularly new items from Approach 1, met or exceeded the quality of existing tests in both expert evaluations and empirical validations, while adapted items showed some limitations. This research demonstrates LLMs’ potential for developing valid assessments, offering insights for future applications.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

AI as a partner in assessment: generating situational judgment tests with large language models

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年11月28日

查看全文
  • Liming Jiang,
  • Fang Luo,
  • Xuetao Tian

摘要

Situational Judgment Tests (SJTs) are important tools for evaluating individual abilities by presenting specific scenarios and assessing the responses. This study explores the potential of Large Language Models (LLMs) in developing SJTs, with emotional regulation as an example. We proposed and evaluated two approaches using ERNIE 4.0, an advanced Chinese LLM: generating new items from scratch (Approach 1) and adapting items from existing examples (Approach 2). The quality of generated items was assessed through expert evaluations and empirical testing with 93 psychology undergraduates and 184 participants from a general population pool. Results indicate that items generated by ERNIE 4.0, particularly new items from Approach 1, met or exceeded the quality of existing tests in both expert evaluations and empirical validations, while adapted items showed some limitations. This research demonstrates LLMs’ potential for developing valid assessments, offering insights for future applications.