Scholarlink

Purpose <p>Differential item functioning (DIF) detection methods are one tool to assess validity of patient-reported outcome measures (PROMs). Given the growing interest in machine-learning models for the analysis of PROMs data, including for DIF detection, internal cross-validation (CV) is recommended to assess model performance. We compared two CV methods for DIF models in simulated and real-world data.</p> Methods <p>Two common CV methods, <i>k</i>-fold and holdout CV, were compared for a machine-learning recursive partitioning model to detect DIF. Simulation parameters included sample size, number of items and covariates associated with DIF, and DIF effect size. Simulation performance was assessed using true positive rate (TPR; power) and false positive rate (FPR; Type I error) for items, covariates, and all item-covariate combinations. DIF performance was also assessed in real-world Hospital Anxiety and Depression Scale (HADS) item responses using area under the receiver operating characteristic (AUROC) curve.</p> Results <p>In simulations, TPRs for consistently identifying items and covariates associated with DIF increased as DIF effect size, sample size, and number of items increased for both CV methods. The 10-fold CV method had the highest TPR in many conditions reaching nearly 100% at the item-covariate level while the 50% holdout CV method had the lowest FPR. For HADS items, DIF detection varied between CV methods, with AUROC ranging from 0.74 to 0.80.</p> Conclusion <p>The choice of a CV method for a machine-learning model to detect DIF affected consistency of DIF item detection. The 10-fold CV method was advantageous for balancing systematic error and data stability.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Internal validation methods to detect patient characteristics associated with differential item functioning in patient-reported outcome measures

  • 期刊论文

  • 发表日期：

    2026年4月1日

查看全文
  • Muditha Bodawatte Gedara,
  • Depeng Jiang,
  • Sunmee Kim,
  • Ridwan Sanusi,
  • Tolulope Sajobi,
  • Lisa M. Lix

摘要

Purpose

Differential item functioning (DIF) detection methods are one tool to assess validity of patient-reported outcome measures (PROMs). Given the growing interest in machine-learning models for the analysis of PROMs data, including for DIF detection, internal cross-validation (CV) is recommended to assess model performance. We compared two CV methods for DIF models in simulated and real-world data.

Methods

Two common CV methods, k-fold and holdout CV, were compared for a machine-learning recursive partitioning model to detect DIF. Simulation parameters included sample size, number of items and covariates associated with DIF, and DIF effect size. Simulation performance was assessed using true positive rate (TPR; power) and false positive rate (FPR; Type I error) for items, covariates, and all item-covariate combinations. DIF performance was also assessed in real-world Hospital Anxiety and Depression Scale (HADS) item responses using area under the receiver operating characteristic (AUROC) curve.

Results

In simulations, TPRs for consistently identifying items and covariates associated with DIF increased as DIF effect size, sample size, and number of items increased for both CV methods. The 10-fold CV method had the highest TPR in many conditions reaching nearly 100% at the item-covariate level while the 50% holdout CV method had the lowest FPR. For HADS items, DIF detection varied between CV methods, with AUROC ranging from 0.74 to 0.80.

Conclusion

The choice of a CV method for a machine-learning model to detect DIF affected consistency of DIF item detection. The 10-fold CV method was advantageous for balancing systematic error and data stability.