Predicting Response Latencies on Test Questions Based on Features of the Questions

Estimating the amount of time students spend on a given test question has many benefits. For example, it can be used as a guide on how much time to allocate for an assessment, or as a valuable tool in gauging student effort on a question. The goal of the present study is to identify the characteristics of a written test question (e.g., coverage of topics within the knowledge domain, use of content vocabulary, linguistic features, and question design features) that were most likely to predict median response times for each test question. The dataset consisted of 610 Advanced Placement statistics questions, and the following regression models were constructed: Stepwise Forward and Backward regression, Lasso regression, and a regression tree. Our findings suggest that: (a) variables pertaining to linguistic features, cognitive complexity, and question design all play a significant role in predicting response latency; and (b) these variables collectively explain a substantial amount of variation in response time (~25%).