Introduction The heuristic shrinkage factor of Van Houwelingen and Le Cessie (𝑆𝑉𝐻) is a commonly used closed-form solution to adjust for overfitting in unpenalised logistic regression models for risk prediction. It is also the basis of widely-adopted minimum sample size criteria for developing clinical prediction models. However, current evidence is lacking regarding the bias of 𝑆𝑉𝐻 compared to the optimal shrinkage factor (𝑆𝑜𝑝𝑡). Here, we examine this issue and also assess the bias of an alternative bootstrap-derived shrinkage factor (𝑆𝑏𝑜𝑜𝑡). Methods We undertook two simulation studies. The first examined the bias of 𝑆𝑉𝐻 and 𝑆𝑏𝑜𝑜𝑡 as estimators of 𝑆𝑜𝑝𝑡 across a range of different scenarios defined by 𝐶𝑝𝑜𝑝, the C-statistic of the model developed in a population sized dataset. The second examined the bias of 𝑆𝑜𝑝𝑡 when using development sample sizes targeting a shrinkage of 0.9, based on a sample size calculation defined by 𝑆𝑉𝐻 itself (𝑁𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙) or by an adapted simulation-based approach (𝑁𝑠𝑖𝑚). Results For high C-statistics, 𝑆𝑉𝐻 overestimated 𝑆𝑜𝑝𝑡, whereas for low C-statistics 𝑆𝑉𝐻 underestimates 𝑆𝑜𝑝𝑡. For example, across scenarios when 0.8≤𝐶𝑝𝑜𝑝<0.85, the 95-percentile range in the bias was (0.005,0.387), compared to (−0.580,−0.007) across scenarios when 0.6≤𝐶𝑝𝑜𝑝<0.65. The magnitude of bias increased as 𝐶𝑝𝑜𝑝 tended to either 0.5 or 1. As sample size increased and 𝑆𝑜𝑝𝑡→1, the magnitude of the bias in either direction reduced. 𝑆𝑏𝑜𝑜𝑡 was less biased than 𝑆𝑉𝐻, with a median magnitude of bias across all scenarios of 0.007, compared to 0.032 for 𝑆𝑉𝐻. Developing models on datasets of size 𝑁𝑠𝑖𝑚 gave 𝑚𝑒𝑎𝑛(𝑆𝑜𝑝𝑡) closer to 0.9 (mean magnitude of bias across all scenarios 0.004) than 𝑁𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙 (mean magnitude of bias 0.041). Conclusions 𝑆𝑉𝐻 is often a poor estimator of the optimal global shrinkage factor. If global shrinkage is needed, we recommend using the bootstrap shrinkage estimate. The bootstrap estimate shows minimal bias in most scenarios, though in small samples the variability is large so provides no guarantees to address overfitting in a single dataset. A sample size calculation based on simulation is often preferable over formula dependent on targeting 𝑆𝑉𝐻.

Agreement between heuristic shrinkage factor and optimal shrinkage factors in logistic regression for risk prediction: a simulation study across different sample sizes and settings

期刊论文
出版模式：

开放获取
发表日期：

2026年5月18日

查看全文

Alexander Pate,
Glen P. Martin,
Richard D. Riley

摘要

Introduction

The heuristic shrinkage factor of Van Houwelingen and Le Cessie (𝑆_𝑉𝐻) is a commonly used closed-form solution to adjust for overfitting in unpenalised logistic regression models for risk prediction. It is also the basis of widely-adopted minimum sample size criteria for developing clinical prediction models. However, current evidence is lacking regarding the bias of 𝑆_𝑉𝐻 compared to the optimal shrinkage factor (𝑆_𝑜𝑝𝑡). Here, we examine this issue and also assess the bias of an alternative bootstrap-derived shrinkage factor (𝑆_{𝑏𝑜𝑜𝑡}).

Methods

We undertook two simulation studies. The first examined the bias of 𝑆_𝑉𝐻 and 𝑆_{𝑏𝑜𝑜𝑡} as estimators of 𝑆_𝑜𝑝𝑡 across a range of different scenarios defined by 𝐶_𝑝𝑜𝑝, the C-statistic of the model developed in a population sized dataset. The second examined the bias of 𝑆_𝑜𝑝𝑡 when using development sample sizes targeting a shrinkage of 0.9, based on a sample size calculation defined by 𝑆_𝑉𝐻 itself (𝑁_{𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙}) or by an adapted simulation-based approach (𝑁_𝑠𝑖𝑚).

Results

For high C-statistics, 𝑆_𝑉𝐻 overestimated 𝑆_𝑜𝑝𝑡, whereas for low C-statistics 𝑆_𝑉𝐻 underestimates 𝑆_𝑜𝑝𝑡. For example, across scenarios when 0.8≤𝐶_𝑝𝑜𝑝<0.85, the 95-percentile range in the bias was (0.005,0.387), compared to (−0.580,−0.007) across scenarios when 0.6≤𝐶_𝑝𝑜𝑝<0.65. The magnitude of bias increased as 𝐶_𝑝𝑜𝑝 tended to either 0.5 or 1. As sample size increased and 𝑆_𝑜𝑝𝑡→1, the magnitude of the bias in either direction reduced. 𝑆_{𝑏𝑜𝑜𝑡} was less biased than 𝑆_𝑉𝐻, with a median magnitude of bias across all scenarios of 0.007, compared to 0.032 for 𝑆_𝑉𝐻. Developing models on datasets of size 𝑁_𝑠𝑖𝑚 gave 𝑚𝑒𝑎𝑛(𝑆_𝑜𝑝𝑡) closer to 0.9 (mean magnitude of bias across all scenarios 0.004) than 𝑁_{𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙} (mean magnitude of bias 0.041).

Conclusions

𝑆_𝑉𝐻 is often a poor estimator of the optimal global shrinkage factor. If global shrinkage is needed, we recommend using the bootstrap shrinkage estimate. The bootstrap estimate shows minimal bias in most scenarios, though in small samples the variability is large so provides no guarantees to address overfitting in a single dataset. A sample size calculation based on simulation is often preferable over formula dependent on targeting 𝑆_𝑉𝐻.