Scoring and Scaling Items from Innovative Domains
摘要
In recent years, there is an increasing interest in measuring more complex and innovative constructs through interactive and gamified tasks. To guide scoring, scaling, and reporting aspects of such innovative constructs in international large-scale assessments, this chapter starts with illustrating the data collected from innovative tasks as human-computer interactions. We then explain the general approach to understand and analyze data from innovative domains and illustrate two prominent examples: automated scoring of text sequence responses and feature generation of PISA 2015 collaborative problem solving (CPS) items. We conclude the chapter with the cautionary note on analyzing and interpreting the results of innovative domains with respect to the reliability, validity, and comparability of the assessment.