An empirical comparison of statistical methods for multiple cut-off diagnostic test accuracy meta-analysis of the Edinburgh postnatal depression scale (EPDS) depression screening tool using published results vs individual participant data

期刊论文
出版模式：

开放获取
发表日期：

2024年2月1日

查看全文

Zelalem F. Negeri,
Brooke Levis,
John P. A. Ioannidis,
Brett D. Thombs,
Andrea Benedetti,
Ying Sun,
Chen He,
Ankur Krishnan,
Yin Wu,
Parash Mani Bhandari,
Dipika Neupane,
Mahrukh Imran,
Danielle B. Rice,
Marleine Azar,
Matthew J. Chiovitti,
Kira E. Riehm,
Jill T. Boruff,
Pim Cuijpers,
Simon Gilbody,
Lorie A. Kloda,
Scott B. Patten,
Roy C. Ziegelstein,
Sarah Markham,
Liane Comeau,
Nicholas D. Mitchell,
Simone N. Vigod,
Muideen O. Bakare,
Cheryl Tatano Beck,
Adomas Bunevicius,
Tiago Castro e Couto,
Genesis Chorwe-Sungani,
Nicolas Favez,
Sally Field,
Lluïsa Garcia-Esteve,
Simone Honikman,
Dina Sami Khalifa,
Jane Kohlhoff,
Laima Kusminskas,
Zoltán Kozinszky,
Sandra Nakić Radoš,
Susan J. Pawlby,
Tamsen J. Rochat,
Deborah J. Sharp,
Johanne Smith-Nielsen,
Kuan-Pin Su,
Meri Tadinac,
S. Darius Tandon,
Pavaani Thiagayson,
Annamária Töreki,
Anna Torres-Giménez,
Thandi van Heyningen,
Johann M. Vega-Dienstmaier

摘要

Background

Selective reporting of results from only well-performing cut-offs leads to biased estimates of accuracy in primary studies of questionnaire-based screening tools and in meta-analyses that synthesize results. Individual participant data meta-analysis (IPDMA) of sensitivity and specificity at each cut-off via bivariate random-effects models (BREMs) can overcome this problem. However, IPDMA is laborious and depends on the ability to successfully obtain primary datasets, and BREMs ignore the correlation between cut-offs within primary studies.

Methods

We compared the performance of three recent multiple cut-off models developed by Steinhauser et al., Jones et al., and Hoyer and Kuss, that account for missing cut-offs when meta-analyzing diagnostic accuracy studies with multiple cut-offs, to BREMs fitted at each cut-off. We used data from 22 studies of the accuracy of the Edinburgh Postnatal Depression Scale (EPDS; 4475 participants, 758 major depression cases). We fitted each of the three multiple cut-off models and BREMs to a dataset with results from only published cut-offs from each study (published data) and an IPD dataset with results for all cut-offs (full IPD data). We estimated pooled sensitivity and specificity with 95% confidence intervals (CIs) for each cut-off and the area under the curve.

Results

Compared to the BREMs fitted to the full IPD data, the Steinhauser et al., Jones et al., and Hoyer and Kuss models fitted to the published data produced similar receiver operating characteristic curves; though, the Hoyer and Kuss model had lower area under the curve, mainly due to estimating slightly lower sensitivity at lower cut-offs. When fitting the three multiple cut-off models to the full IPD data, a similar pattern of results was observed. Importantly, all models had similar 95% CIs for sensitivity and specificity, and the CI width increased with cut-off levels for sensitivity and decreased with an increasing cut-off for specificity, even the BREMs which treat each cut-off separately.

Conclusions

Multiple cut-off models appear to be the favorable methods when only published data are available. While collecting IPD is expensive and time consuming, IPD can facilitate subgroup analyses that cannot be conducted with published data only.