Quantifying Uncertainty in Regression-Based Norms

Normative data essentially allow for converting a raw test score into a percentile rank. The obtained percentile ranks are estimates of the true (population-level) relative position of a test score. Indeed, there is always uncertainty in the estimated percentile ranks because the population distribution of the raw test scores is unknown. This chapter introduces a bootstrap procedure that allows for quantifying the uncertainty in the percentile ranks (in the form of Confidence Intervals; CIs). The approach is fully general and can account for violations of the model assumptions (should these occur). A simulation study is conducted to evaluate the performance of the proposed bootstrap approach. The results show that the coverage of the percentile bootstrap CIs is good, i.e., the \(95\%\) / \(99\%\) CIs around the point estimates of the percentile ranks contain the true percentile ranks in approximately \(95\%\) / \(99\%\) of the cases in the different simulation scenarios. Further, it is illustrated how a simulation-based approach can also be used to conduct sample size estimations for normative studies. The methodology to derive the percentile bootstrap CIs is exemplified in two case studies that were already analyzed in previous chapters, i.e., for the Science Exam subscale score of the General Certificate of Secondary Education test and the Letter Digit Substitution Test score. The analyses are conducted using the NormData package in the R software.