Scholarlink

<p>Machine learning models that predict hospital admission at triage may support patient flow forecasting, yet the effects of covariate drift, concept drift, and retraining on long-term performance are poorly understood. We developed an Extreme Gradient Boosting (XGBoost) model using deidentified data from all presentations to a metropolitan hospital in Western Australia. Training and validation included 2016 and 2017 presentations (<i>n</i> = 133,814), with rolling quarterly testing from 2018 to 2023 (<i>n</i> = 455,496). Two adaptive strategies were evaluated: quarterly and half-yearly retraining. Covariate drift was assessed using univariate and multivariable analyses, and reporting adhered to TRIPOD + AI and MINIMAR standards. Substantial drift was observed both between training and testing datasets and across the six-year testing period. The base model achieved a mean AUROC of 0.875 (range 0.844–0.887) and mean daily bed error of 7.42 beds (range 0.47–13.1). Retrained models demonstrated improved discrimination (mean AUROC 0.892 and 0.893) and reduced bed error (4.02 and 4.48 beds per day) for quarterly and half-yearly retraining, respectively, with similar calibration and classification performance. Covariate drift meaningfully degraded calibration but not discrimination metrics over time. Simple retraining improved discrimination and reduced calibration concept drift, underscoring the importance of retraining to address drift for temporal model deployment.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Simulation of covariate and concept drift in machine learning hospital admission prediction from emergency triage

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月10日

查看全文
  • Ethan Williams,
  • Toshi Sinha,
  • Matthew Summerscales,
  • Yogesan Kanagasingam

摘要

Machine learning models that predict hospital admission at triage may support patient flow forecasting, yet the effects of covariate drift, concept drift, and retraining on long-term performance are poorly understood. We developed an Extreme Gradient Boosting (XGBoost) model using deidentified data from all presentations to a metropolitan hospital in Western Australia. Training and validation included 2016 and 2017 presentations (n = 133,814), with rolling quarterly testing from 2018 to 2023 (n = 455,496). Two adaptive strategies were evaluated: quarterly and half-yearly retraining. Covariate drift was assessed using univariate and multivariable analyses, and reporting adhered to TRIPOD + AI and MINIMAR standards. Substantial drift was observed both between training and testing datasets and across the six-year testing period. The base model achieved a mean AUROC of 0.875 (range 0.844–0.887) and mean daily bed error of 7.42 beds (range 0.47–13.1). Retrained models demonstrated improved discrimination (mean AUROC 0.892 and 0.893) and reduced bed error (4.02 and 4.48 beds per day) for quarterly and half-yearly retraining, respectively, with similar calibration and classification performance. Covariate drift meaningfully degraded calibration but not discrimination metrics over time. Simple retraining improved discrimination and reduced calibration concept drift, underscoring the importance of retraining to address drift for temporal model deployment.