Item Response Theory Modeling with Response Times: Some Issues

The increased prevalence of item response time (RT) data along with item responses has made applications of several joint item response theory (IRT) models feasible. Molenaar et al. (Multivar Behav Res 50: 56–74, 2015) unified several IRT models with joint response accuracy and RT into a common hierarchical framework to possibly increase the measurement precision of trait. Depending on the model, the assumed relationship between response accuracy and response time may be either positive or negative. However, a previous study (Embretson, 2021) found that examinees differed substantially in the relationship of their item response times to both item difficulty and test position on a spatial ability test. Such differences could impact the advantages of the various joint models. In this chapter, two additional types of tests, analytic reasoning and mathematics achievement, were studied for examinee differences in their item response time relationships within the test. Thus, for each examinee, their item response times will be correlated with both item difficulty and item test position. Broad distributions of these examinee correlations, ranging from strong negative to strong positive, were found for both tests. Hence, models that assume a uniform relationship of item response time to item difficulty across examinees may not be appropriate for many types of tests.