Scholarlink

<p>Software fault prediction (SFP) models are often trained on severely imbalanced data. Prior work also warns that low events‑per‑variable (EPV) ratios can yield unstable and biased estimates. To quantify the separate and joint impact of (i) class‑balancing techniques and (ii) EPV ratios on the accuracy and stability of SFP models trained on five real‑world, open‑source GitHub projects. We also examine whether classifier choice outweighs either factor. We extracted five projects from GitHub and built class‑level datasets. After feature selection (genetic algorithm), we evaluated five estimation techniques (PLS, CART, NB, RF, SVM) under a factorial design crossing EPV ∈ (1) with balancing ∈ (None, Up, Down, SMOTE, ROSE) [1]. AUC is used to measure performance. We report effect sizes and stability (across repeated runs) and profile computational cost. Baselines include (a) no balancing at each EPV level and (b) an additional “raw” baseline without EPV adjustment for the only dataset with non‑zero defects (Ceylon). (i) Balancing improves AUC in a minority of settings; SMOTE is the most reliable of the balancing methods, whereas ROSE is the least effective. (ii) EPV = 20 consistently yields higher accuracy and stability than EPV = 3 or 10. (iii) Classifier choice matters most: strong estimation models (RF/SVM) at low EPV often outperform weak estimation techniques (PLS/CART) at high EPV. For modern GitHub‑based SFP, increasing EPV and choosing an appropriate classifier have a greater impact than applying generic class‑balancing. We provide a reproducible protocol (data processing pipeline, seeds, and scripts) and outline an external validation plan on NASA, PROMISE, and Relink.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Impact of class balancing techniques or events per variable ratio on the conclusion of the software faults prediction models

  • 期刊论文

  • 发表日期：

    2025年11月24日

查看全文
  • Asad Ali,
  • Dost Muhammad,
  • Mushtaq Ali,
  • Fakhrud Din

摘要

Software fault prediction (SFP) models are often trained on severely imbalanced data. Prior work also warns that low events‑per‑variable (EPV) ratios can yield unstable and biased estimates. To quantify the separate and joint impact of (i) class‑balancing techniques and (ii) EPV ratios on the accuracy and stability of SFP models trained on five real‑world, open‑source GitHub projects. We also examine whether classifier choice outweighs either factor. We extracted five projects from GitHub and built class‑level datasets. After feature selection (genetic algorithm), we evaluated five estimation techniques (PLS, CART, NB, RF, SVM) under a factorial design crossing EPV ∈ (1) with balancing ∈ (None, Up, Down, SMOTE, ROSE) [1]. AUC is used to measure performance. We report effect sizes and stability (across repeated runs) and profile computational cost. Baselines include (a) no balancing at each EPV level and (b) an additional “raw” baseline without EPV adjustment for the only dataset with non‑zero defects (Ceylon). (i) Balancing improves AUC in a minority of settings; SMOTE is the most reliable of the balancing methods, whereas ROSE is the least effective. (ii) EPV = 20 consistently yields higher accuracy and stability than EPV = 3 or 10. (iii) Classifier choice matters most: strong estimation models (RF/SVM) at low EPV often outperform weak estimation techniques (PLS/CART) at high EPV. For modern GitHub‑based SFP, increasing EPV and choosing an appropriate classifier have a greater impact than applying generic class‑balancing. We provide a reproducible protocol (data processing pipeline, seeds, and scripts) and outline an external validation plan on NASA, PROMISE, and Relink.