Assessing Individual Change: A Comparison of Reliable Change Indices Based on Classical Test Theory and Various Item Response Theory Models

For the evaluation of individual change during an intervention, several methods have been proposed that are based on classical test theory (CTT), in particular, the reliable change index (RCI) of Jacobson and Truax (J Consult Clin Psychol 59:12–19, 1991). In addition, RCIs based on item response theory (IRT) have been suggested which seem superior since IRT models imply varying standard errors over the trait. In the present chapter, we give a formal introduction to the assessment of individual change based on these two approaches and discuss studies that compare these two in simulation and empirical studies. Then, these methods were applied to a data set of inpatients (N = 7276), assessed with the self-reported severity of depression (BDI-II) at admission and at discharge. In addition to existing studies, we tested different IRT models, compared their goodness of fit, and examined the assumption of longitudinal invariance. Some main results were (i) the IRT-based RCIs showed the expected biconcave borders of significant changes, in contrast to the straight lines resulting from the classical RCI; (ii) about 60% of the patients improved from a statistical point of view and less than 2% deteriorated; (iii) the overall agreement between classical RCI with conventional IRT comes to 94% and with two-dimensional IRT 90%; and (iv) however, there were category-specific misclassification rates up to 10–15% between classical and IRT-based RCI. Results are discussed with regard to which models are most preferable and in the context of the broader concept of “meaningful change.”