Evaluation

The purpose of model evaluation is to assess the model’s suitability for the intended purpose. In the evaluation of clinical models, we consider three levels of evaluation. At the core, we are concerned with predictive performance, namely whether the model we constructed has sufficiently high predictive ability. On the next level, we are concerned with generalizability. We wish to ensure the model is robust to changes over time and we may wish to know whether the model can generalize to different demographics at different geographic locations or to a different service with different disease severity. Finally, on the third level, we evaluate the model from the perspective of achieving the clinical objective and doing so at a cost that is acceptable to the health system.