Background <p>The net benefit and decision curve analysis are increasingly being used to assess the clinical utility of prognostic models. This metric assesses the value added by a model’s predictions when individuals are treated differently according to whether they are over or under a chosen threshold. Although such ‘treat or not’ decisions are common, prognostic models are also often used to tailor and personalise the care of patients, which implicitly involves the consideration of multiple interventions at different risk thresholds. We aim to extend decision curve analysis to estimate the net benefit of a model over multiple thresholds.</p> Methods <p>We take a weighted area under a rescaled version of the net benefit curve, deriving the continuous net benefit. In addition to the consideration of a continuum of interventions, we also show how the continuous net benefit can be used to evaluate single treatments in populations with a range of optimal thresholds, due to individual variations in expected treatment benefit or harm, highlighting limitations of current proposed methods that calculate the area under the decision curve. We propose this not as a substitute for decision curves, but as a complementary evaluation metric, in lieu of single-threshold point estimates.</p> Results <p>We showcase this metric through two examples of model validation in cardiovascular preventive care. The continuous net benefit brought additional insight over point estimates when comparing models over a range of decisions.</p> Conclusions <p>The continuous net benefit informs those looking to validate clinical prediction models of their clinical utility, and helps decision makers understand their usefulness, improving their viability towards implementation.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

The continuous net benefit: assessing the clinical utility of prediction models when informing a continuum of decisions

  • Jose Benitez-Aurioles,
  • Laure Wynants,
  • Niels Peek,
  • Patrick Goodley,
  • Philip Crosbie,
  • Matthew Sperrin

摘要

Background

The net benefit and decision curve analysis are increasingly being used to assess the clinical utility of prognostic models. This metric assesses the value added by a model’s predictions when individuals are treated differently according to whether they are over or under a chosen threshold. Although such ‘treat or not’ decisions are common, prognostic models are also often used to tailor and personalise the care of patients, which implicitly involves the consideration of multiple interventions at different risk thresholds. We aim to extend decision curve analysis to estimate the net benefit of a model over multiple thresholds.

Methods

We take a weighted area under a rescaled version of the net benefit curve, deriving the continuous net benefit. In addition to the consideration of a continuum of interventions, we also show how the continuous net benefit can be used to evaluate single treatments in populations with a range of optimal thresholds, due to individual variations in expected treatment benefit or harm, highlighting limitations of current proposed methods that calculate the area under the decision curve. We propose this not as a substitute for decision curves, but as a complementary evaluation metric, in lieu of single-threshold point estimates.

Results

We showcase this metric through two examples of model validation in cardiovascular preventive care. The continuous net benefit brought additional insight over point estimates when comparing models over a range of decisions.

Conclusions

The continuous net benefit informs those looking to validate clinical prediction models of their clinical utility, and helps decision makers understand their usefulness, improving their viability towards implementation.