Data balancing improves mortality prediction for emergency department patients

Accurate patient mortality prediction is crucial in the emergency department (ED) to improve emergency healthcare services. Current prediction models are limited in both accuracy and practicality, particularly in identifying high-risk patients early. Machine Learning is deeply affected by data quality because mortality samples are significantly fewer than survival samples. This study aimed to achieve balanced and better accuracy of patient mortality prediction and evaluate the effectiveness of data balancing methods.

This study analyzed 2,437,341 non-traumatic adult ED visit records collected between 2008 and 2016 from five medical centers in Taiwan, including four mortality timeframes: death within 24, 72, and 168 h, and final death, and evaluated three data balancing methods: Random Under Sampling (RUS), Synthesized Minority Oversampling Technique (SMOTE), and Random Over Sampling (ROS). We adopted Random Forest (RF), AdaBoost (ADA), XG Boost (XGB). Logistic Regression (LR) is the meta learner for these models. Besides, we performed feature importance analysis based on RF, ADA, AdaBoost with BootStrap (ADA-BS), and Information Gain (IG).

Our model with XGB achieved the best AUROC, 91.41%, which is better than 90.2% in the previous study by Wu et al. using the same dataset in 168-hour mortality timeframe. Our True Positive Rate (TPR) and True Negative Rate (TNR) are 79.88% and 86.73%, which are more balanced than 25% and 100% in the previous study. ROS achieves the better results than RUS and SMOTE and becomes our primary data balancing method. While adopting XGB in 24-hour mortality timeframes, ROS achieved the best AUROC, 93.72%, RUS achieved 93.61% and SMOTE achieved 91.73%. Compared with the previous study by Lin et al., the feature importance analysis shows our balanced dataset has better feature importance impacts, especially for the “Age” and “Triage” features.

Our method achieves better AUROC than the previous study, especially in the long challenging death-hour mortality timeframe with XGB and ROS. Our method achieves balanced TPR and TNR, which are more practical than AUROC. Besides, feature importance analysis shows our balanced dataset has better feature importance impacts.