Background <p>The early diagnosis of hepatorenal syndrome (HRS) is constrained by the reliance on serum creatinine, a biomarker with well-documented limitations in sensitivity and timeliness, contributing to diagnostic delays and adverse outcomes. Machine learning (ML) offers a potential solution, but its translation is hindered by two critical gaps: the absence of prospective, head-to-head validation against the clinical standard, and inadequate assessment of model robustness against temporal data distribution shift—a pivotal challenge for real-world deployment.</p> Methods <p>We developed an XGBoost model using a retrospective cohort of patients with decompensated cirrhosis (<i>n</i> = 464). Its performance was then prospectively validated in a completely independent, temporally distinct cohort (<i>n</i> = 269) in a direct comparison against the serum creatinine diagnostic standard. The prospective cohort was further split into sequential subsets to assess short-term internal temporal robustness. The evaluation framework encompassed diagnostic performance, interpretability (SHAP analysis), clinical utility (decision curve analysis), and health economic evaluation.</p> Results <p>The model demonstrated exceptional discriminatory accuracy, with an area under the receiver operating characteristic curve (AUC) of 0.994 (95% bias-corrected and accelerated bootstrap CI: 0.985–0.998). This represented a substantial and statistically significant improvement over serum creatinine-based diagnosis (<i>AUC</i> = 0.803; <i>p</i> &lt; 0.001). The model’s predictions were well-calibrated, and its performance remained stable in the limited internal temporal robustness assessment. The health economic evaluation established the model’s decisive cost-effectiveness, with an incremental cost-effectiveness ratio (ICER) of 571 EUR per quality-adjusted life year (QALY), substantially below major international willingness-to-pay thresholds, including Indonesia’s benchmark of 3853 EUR/QALY.</p> Conclusion <p>This study provides three key contributions: (1) conclusive, prospective evidence that an ML model significantly outperforms the current serum creatinine standard for HRS risk stratification; (2) a novel methodological framework for proactively assessing temporal robustness, which demonstrated stability in a short-term, single-center setting; and (3) compelling evidence of the model’s accuracy, clinical utility, and cost-effectiveness within a rigorous, single-center prospective validation. While these results establish a high-fidelity proof of concept, the essential next steps are external validation across multiple centers to confirm generalizability and the assessment of resilience to longer-term, real-world dataset shifts before any consideration of broader clinical implementation.</p> Graphical abstract <p></p> <p>This illustration summarizes the development and validation of a novel XGBoost-based machine learning model for the identifying high-risk patients before serum creatinine meets diagnostic criteria of hepatorenal syndrome (HRS). The model, trained on multimodal clinical data and prospectively validated in an independent temporal cohort, demonstrated superior discriminatory performance (<i>AUC</i> = 0.994) compared to the serum creatinine (sCr) criterion (<i>AUC</i> = 0.803). The model’s implementation is projected to enable the identification of thousands of additional patients and avert numerous deaths, proving to be a highly cost-effective diagnostic strategy.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Machine learning outperforms serum creatinine for early risk stratification of hepatorenal syndrome: a prospective single-center validation

  • Yuli Song,
  • Weixiao Shi,
  • Chengchen Yang,
  • Xiaochen Yang,
  • Chengbo Yu

摘要

Background

The early diagnosis of hepatorenal syndrome (HRS) is constrained by the reliance on serum creatinine, a biomarker with well-documented limitations in sensitivity and timeliness, contributing to diagnostic delays and adverse outcomes. Machine learning (ML) offers a potential solution, but its translation is hindered by two critical gaps: the absence of prospective, head-to-head validation against the clinical standard, and inadequate assessment of model robustness against temporal data distribution shift—a pivotal challenge for real-world deployment.

Methods

We developed an XGBoost model using a retrospective cohort of patients with decompensated cirrhosis (n = 464). Its performance was then prospectively validated in a completely independent, temporally distinct cohort (n = 269) in a direct comparison against the serum creatinine diagnostic standard. The prospective cohort was further split into sequential subsets to assess short-term internal temporal robustness. The evaluation framework encompassed diagnostic performance, interpretability (SHAP analysis), clinical utility (decision curve analysis), and health economic evaluation.

Results

The model demonstrated exceptional discriminatory accuracy, with an area under the receiver operating characteristic curve (AUC) of 0.994 (95% bias-corrected and accelerated bootstrap CI: 0.985–0.998). This represented a substantial and statistically significant improvement over serum creatinine-based diagnosis (AUC = 0.803; p < 0.001). The model’s predictions were well-calibrated, and its performance remained stable in the limited internal temporal robustness assessment. The health economic evaluation established the model’s decisive cost-effectiveness, with an incremental cost-effectiveness ratio (ICER) of 571 EUR per quality-adjusted life year (QALY), substantially below major international willingness-to-pay thresholds, including Indonesia’s benchmark of 3853 EUR/QALY.

Conclusion

This study provides three key contributions: (1) conclusive, prospective evidence that an ML model significantly outperforms the current serum creatinine standard for HRS risk stratification; (2) a novel methodological framework for proactively assessing temporal robustness, which demonstrated stability in a short-term, single-center setting; and (3) compelling evidence of the model’s accuracy, clinical utility, and cost-effectiveness within a rigorous, single-center prospective validation. While these results establish a high-fidelity proof of concept, the essential next steps are external validation across multiple centers to confirm generalizability and the assessment of resilience to longer-term, real-world dataset shifts before any consideration of broader clinical implementation.

Graphical abstract

This illustration summarizes the development and validation of a novel XGBoost-based machine learning model for the identifying high-risk patients before serum creatinine meets diagnostic criteria of hepatorenal syndrome (HRS). The model, trained on multimodal clinical data and prospectively validated in an independent temporal cohort, demonstrated superior discriminatory performance (AUC = 0.994) compared to the serum creatinine (sCr) criterion (AUC = 0.803). The model’s implementation is projected to enable the identification of thousands of additional patients and avert numerous deaths, proving to be a highly cost-effective diagnostic strategy.