The Impact of Generating Model on Preknowledge Detection in CAT

Recent years have seen a growing interest in the development of methods for detecting examinees with preknowledge, especially in the context of computerized adaptive testing (CAT). Because it is difficult to obtain real data in which the examinees with preknowledge and the compromised items are known with absolute certainty, the performance of such methods is typically evaluated using simulation studies where models are used to generate the data. However, with different researchers making different choices regarding which models to use and how the data should be generated, it becomes challenging, if not impossible, to find ways to compare the results of one simulation study to another. In this chapter, we examine the impact of generating model on preknowledge detection in CAT. Results indicate that the use of different generating models has the potential to greatly impact detection results.