Autoencoder-based confidence score for item preknowledge detection

Item compromise and preknowledge pose significant challenges in educational testing, leading to the development of various detection methods. Despite advancements, existing approaches often lack a confidence score to indicate the certainty that the detection result truly corresponds to item preknowledge. Building upon the work of Pan and Wollack (Educ Meas Issues Pract 42:76–98, 2023; PW23), we introduced a confidence score that leverages autoencoders—a deep learning model utilizing both response scores and times. Our simulation studies demonstrate that this confidence score accurately assesses the likelihood of detection results aligning with actual item preknowledge. Notably, the proposed confidence score surpasses the PW23 confidence score in effectiveness, particularly in scenarios with a small magnitude of item preknowledge.