Interpretability and Personalization Aspects in the Development of Clinical Risk Assessment Models

Effective risk stratification at hospital admission or discharge is critical for physicians to develop an appropriate treatment plan for each patient. While the GRACE score is the most applied risk assessment tool in Portugal for patients with Acute Coronary Syndrome, it has some limitations. Machine learning models have shown notable performance in risk prediction, but their “black-box” nature is an obstacle to their adoption in healthcare. The main objective of this study is to develop a comprehensive evaluation framework that can compare the performance and interpretability of machine learning models with GRACE score. Logistic Regression, Naïve Bayes and Decision Trees were selected, as they create, though at different levels, interpretable models. A personalized approach based on decision rules, developed by this research team, was also considered. The interpretability of all models was quantified based on three elements: (i) stability assessment; (ii) 95% CI of geometric mean; (iii) correlation between the features rank of each model and the rank defined by GRACE. The dataset (N = 1544) was provided by a Portuguese Hospital. Personalized approach achieved the highest Gmean (74.72%), the highest correlation (0.83), being stability identical to GRACE. The results suggest that personalized approach has potential to be applied in a clinical scenario given its performance (better than GRACE) and its interpretability (similar to GRACE).