DIF Detection in a Response Time Measure: A Likelihood Ratio Test Method

For assessments used in hiring decisions, it is essential to address discrepancies in measurement across subgroups from which job candidates may be compared for the same position, so that scores and subsequent decisions are fair and valid. Computer-administered employment tests today commonly use objective items that capture a response reflecting a job-relevant construct and a response time. Response times may be used to assess individual speed, to calibrate the difficulty of a speeded test, or for other purposes. In order to improve the fairness and validity of speed measures and time limits, we want to check for parametric differential item functioning (DIF) not only in item responses but also in response times. While many joint models of item responses and response times fall into the category of complex multidimensional systems in which DIF is challenging to interpret, the hierarchical framework proposed by Linden (2006, 2007) isolates response and response time models sufficiently that, to the extent that the framework holds, the possibility of DIF in responses and response times may be evaluated separately. Using this framework, likelihood ratio tests of parametric DIF are applied to a three-parameter form of the lognormal response time model. The lognormal response time model can be fit and interpreted as a transformation of a factor model (Finger and Chuah, 2009). Through this relationship, the accepted procedures of factorial invariance testing suggest an order for nested parameter constraint tests in a DIF sweep, corresponding to sequential tests of loadings, intercepts, and unique variances. This facilitates interpretation of DIF findings.