If At First You Don’t Succeed: Effect of Job Applicant Retesting on the Practical Outcomes of Selection
摘要
Organizations often provide the opportunity for job applicants to retest for a position from which they were rejected. The current study asks: What are the practical benefits and liabilities for organizations that allow applicants to retest for a position multiple times? How reliable should test scores be to prevent unqualified applicants from retesting until they are selected just by chance? Among those not selected, how much change in the mean score between initial test and retest can be expected purely due to measurement error variance? To investigate such questions, the current study conducted a Monte Carlo selection simulation that allowed for retesting on a selection test battery consisting of four commonly used tests (measures of cognitive ability, structured interview, conscientiousness, biodata) under realistic levels of test–retest reliability (from rxx = .70 to .95 for each test). The applicants’ true standing on these measures was held constant. Our simulation results provide specific information on a general intuition: retesting increased the number of underqualified applicants who eventually passed, and this increase became practically meaningful as test–retest reliability decreased. From an overall validity perspective, however, the decrease in mean job performance as a function of applicant retesting was minimal. Also, we found sizable mean score changes across retesting occasions that can meaningfully affect high-stakes selection decisions in typical selection situations. Thus, understanding the psychometric and practical implications of retesting and engaging in efforts to reduce construct-irrelevant changes in observed test scores across test occasions should be major points of consideration to help ensure that personnel selection procedures are more accurate and effective.