Examining predictors of scale-up penalty in preliminary studies of health behavior interventions and subsequent larger trials: a meta-epidemiological review of diverse fields
摘要
Scale-up penalty, a common phenomenon in which the promising effects found in early preliminary studies are substantially reduced when evaluated in a subsequent larger trial, can stall the advancement of health behavior interventions. In obesity-related behavioral interventions, changes to key features between a preliminary study and subsequent larger trials inflate scale-up penalty. The purpose of this study is to examine whether changes in intervention features occur in other behavioral disciplines that utilize a similar developmental continuum wherein smaller-scale preliminary studies inform larger-scale trials, and whether changes in key features inflate scale-up penalty.
MethodsWe conducted a systematic review identifying preliminary studies followed by a larger trial conducted by the same author(s) (i.e., a study pair) in four areas—tobacco/smoking cessation, alcohol use, interpersonal violence, and sexually transmitted diseases. We coded intervention features in the preliminary study and larger trial to capture changes in key study features (e.g., who delivered the intervention). Multi-level meta-regressions estimated the association between the changes to key study features and change in standardized mean difference for health outcomes and calculated scale-up penalty.
ResultsWe identified 222 effects across 69 study pairs of preliminary studies with subsequent larger trials. Fifty-eight study pairs (84%) contained at least one change to a key study feature. Forty-nine study pairs were included in the meta-regressions (k = 154 effects). Changing the delivery agent, outcome measure, intervention duration, or implementation support in subsequent larger trials followed the hypothesized pattern with larger scale-up penalty (i.e., less effective at-scale) relative to study pairs that did not change those features.
ConclusionsMultiple disciplines outside the obesity field make changes to key study features between preliminary studies and subsequent larger trials. Across disciplines, changes to the intervention delivery agent, intervention duration, outcome measurement, and reducing implementation support in subsequent larger trials are associated with inflated scale-up penalty. Recognition of the importance of key study features in preliminary studies and iterative intervention development could improve intervention scale-up.