Test retrofitting with competence-based knowledge space theory: empirical applications with two educational tools
摘要
A critical issue within the educational system is the availability of assessment tools capable of accurately describing the skills mastered by a student and those that are not. Competence-based Knowledge Space Theory (CbKST) is a mathematical theory for the formative assessment of skills that aims to precisely identify the set of skills an individual possesses based on the responses to test items. The current landscape in educational measurement includes lots of good tests designed under theories other than CbKST, which, after appropriate analysis, could be used for skill assessment. Identifying the skills underlying the responses to the items of these tests could enable educational stakeholders to obtain precise feedback about students’ mastery of each skill. This work describes retrofitting of existing tests in the framework of CbKST. In particular, it provides a step-by-step guide useful to identify the skills measured by the test and the way they are associated with the items. Retrofitting is illustrated in practice through applications on empirical data collected using two educational tests. Finally, the information about the test, its items, and students’ skills that can be gained from retrofitting is explored and discussed.