Background Multiple treatment options frequently exist for a single medical condition with no single standard of care (SoC), rendering a classic randomised trial comparing a specific treatment to a control treatment infeasible. A novel design, the personalised randomised controlled trial (PRACTical), allows individualised randomisation lists and borrows information across patient subpopulations to rank treatments against each other without comparison to a SoC. We evaluated standard frequentist analysis with Bayesian analyses, and developed a novel performance measure, utilising the precision in treatment coefficient estimates, for treatment ranking. Methods We simulated trial data to compare four targeted antibiotic treatments for multidrug resistant bloodstream infections as an example. Four patient subgroups were simulated based on different combinations of patient and bacteria characteristics, which required four different randomisation lists with some overlapping treatments. The primary outcome was binary, using 60-day mortality. Treatment effects were derived using frequentist and Bayesian analytical approaches, with logistic multivariable regression. The performance measures were: probability of predicting the true best treatment, and novel proxy variables for power (probability of interval separation) and type I error (probability of incorrect interval separation). Several scenarios with varying treatment effects and sample sizes were compared. Results The Frequentist model and Bayesian model using a strong informative prior, were both likely to predict the true best treatment (<InlineEquation ID="IEq1"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="12874_2025_2537_Article_IEq1.gif" Format="GIF" Height="17" Rendition="HTML" Resolution="72" Type="Linedraw" Width="85" /> </InlineMediaObject> <EquationSource Format="TEX">${P}_{best}\ge 80\%$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <msub> <mi>P</mi> <mrow> <mi mathvariant="italic">best</mi> </mrow> </msub> <mo>≥</mo> <mn>80</mn> <mo>%</mo> </mrow> </math></EquationSource> </InlineEquation>) and gave a large probability of interval separation (reaching a maximum of <InlineEquation ID="IEq2"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="12874_2025_2537_Article_IEq2.gif" Format="GIF" Height="17" Rendition="HTML" Resolution="72" Type="Linedraw" Width="78" /> </InlineMediaObject> <EquationSource Format="TEX">${P}_{IS}=96\%$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <msub> <mi>P</mi> <mrow> <mi mathvariant="italic">IS</mi> </mrow> </msub> <mo>=</mo> <mn>96</mn> <mo>%</mo> </mrow> </math></EquationSource> </InlineEquation>), at a given sample size. Both methods had a low probability of incorrect interval separation (<InlineEquation ID="IEq3"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="12874_2025_2537_Article_IEq3.gif" Format="GIF" Height="16" Rendition="HTML" Resolution="72" Type="Linedraw" Width="84" /> </InlineMediaObject> <EquationSource Format="TEX">${P}_{IIS}<0.05$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <msub> <mi>P</mi> <mrow> <mi mathvariant="italic">IIS</mi> </mrow> </msub> <mo><</mo> <mn>0.05</mn> </mrow> </math></EquationSource> </InlineEquation>), for all sample sizes (<InlineEquation ID="IEq4"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="12874_2025_2537_Article_IEq4.gif" Format="GIF" Height="15" Rendition="HTML" Resolution="72" Type="Linedraw" Width="115" /> </InlineMediaObject> <EquationSource Format="TEX">$N=500-5000$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>N</mi> <mo>=</mo> <mn>500</mn> <mo>-</mo> <mn>5000</mn> </mrow> </math></EquationSource> </InlineEquation>) in the null scenarios considered. The sample size required for probability of interval separation to reach 80% (<InlineEquation ID="IEq5"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="12874_2025_2537_Article_IEq5.gif" Format="GIF" Height="15" Rendition="HTML" Resolution="72" Type="Linedraw" Width="123" /> </InlineMediaObject> <EquationSource Format="TEX">$N=1500-3000$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>N</mi> <mo>=</mo> <mn>1500</mn> <mo>-</mo> <mn>3000</mn> </mrow> </math></EquationSource> </InlineEquation>), was larger than the sample size required for the probability of predicting the true best treatment to reach 80% (<InlineEquation ID="IEq6"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="12874_2025_2537_Article_IEq6.gif" Format="GIF" Height="16" Rendition="HTML" Resolution="72" Type="Linedraw" Width="64" /> </InlineMediaObject> <EquationSource Format="TEX">$N\le 500$</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>N</mi> <mo>≤</mo> <mn>500</mn> </mrow> </math></EquationSource> </InlineEquation>). Conclusions Utilising uncertainty intervals on the treatment coefficient estimates are highly conservative, limiting applicability to large pragmatic trials. Bayesian analysis performed similarly to the frequentist approach in terms of predicting the true best treatment.

A comparison of frequentist and Bayesian approaches to the Personalised Randomised Controlled Trial (PRACTical)—design and analysis considerations

期刊论文
出版模式：

开放获取
发表日期：

2025年5月29日

查看全文

Holly Jackson,
Yiyun Shou,
Nur Amira Binte Mohamed Azad,
Jing Wen Chua,
Rebecca Lynn Perez,
Xinru Wang,
Marlieke E. A. de Kraker,
Yin Mo

摘要

Background

Multiple treatment options frequently exist for a single medical condition with no single standard of care (SoC), rendering a classic randomised trial comparing a specific treatment to a control treatment infeasible. A novel design, the personalised randomised controlled trial (PRACTical), allows individualised randomisation lists and borrows information across patient subpopulations to rank treatments against each other without comparison to a SoC. We evaluated standard frequentist analysis with Bayesian analyses, and developed a novel performance measure, utilising the precision in treatment coefficient estimates, for treatment ranking.

Methods

We simulated trial data to compare four targeted antibiotic treatments for multidrug resistant bloodstream infections as an example. Four patient subgroups were simulated based on different combinations of patient and bacteria characteristics, which required four different randomisation lists with some overlapping treatments. The primary outcome was binary, using 60-day mortality. Treatment effects were derived using frequentist and Bayesian analytical approaches, with logistic multivariable regression. The performance measures were: probability of predicting the true best treatment, and novel proxy variables for power (probability of interval separation) and type I error (probability of incorrect interval separation). Several scenarios with varying treatment effects and sample sizes were compared.

Results

The Frequentist model and Bayesian model using a strong informative prior, were both likely to predict the true best treatment ( ${P}_{best}\ge 80\%$ $P_{best} \geq 80 %$ ) and gave a large probability of interval separation (reaching a maximum of ${P}_{IS}=96\%$ $P_{IS} = 96 %$ ), at a given sample size. Both methods had a low probability of incorrect interval separation ( ${P}_{IIS}<0.05$ $P_{IIS} < 0.05$ ), for all sample sizes ( $N=500-5000$ $N = 500 - 5000$ ) in the null scenarios considered. The sample size required for probability of interval separation to reach 80% ( $N=1500-3000$ $N = 1500 - 3000$ ), was larger than the sample size required for the probability of predicting the true best treatment to reach 80% ( $N\le 500$ $N \leq 500$ ).

Conclusions

Utilising uncertainty intervals on the treatment coefficient estimates are highly conservative, limiting applicability to large pragmatic trials. Bayesian analysis performed similarly to the frequentist approach in terms of predicting the true best treatment.