In many domains individuals have to acquire and retain a large amount of domain-specific knowledge and skills. Training can be expensive and time consuming but reaching and maintaining proficiency is often vital. To decrease training time and increase retention, adaptive instructional systems (AIS; e.g., intelligent tutoring systems) can individualize a curriculum for students. To accomplish this, various models of learning and retention have been implemented into AISs. Although these types of models have been successfully applied across different contexts these models assume that performance is generated by a single continuous underlying mechanism, making it difficult to account for highly variable individual performance. To account for variable individual performance, models of learning and retention can be paired with change detection algorithms (CDA) to detect homogeneous segments of performance. In this paper, we conduct a model simulation study using four different models of learning and retention comparing three different CDAs’ ability to fit, infer the underlying model, and detect change points across multiple simulated conditions.