This study evaluates the efficacy of Predictive Mutation Testing (PMT) models, focusing on the impact of preprocessing, class rebalancing, and model selection on predictive accuracy. By filtering unreached mutants, artificially inflated metrics caused by trivial correlations were minimized, enhancing the ability of the model to identify complex patterns. While ensemble models like CatBoost and XGBoost showed high effectiveness in detecting killed mutants, their performance in identifying survived mutants was limited, underscoring the need for refined strategies to address class imbalance. The study further identifies that previous PMT research often lacks class-specific metrics, such as the Matthews Correlation Coefficient, essential for assessing balanced performance. This oversight can yield misleading evaluations by favoring the majority class, which may falsely indicate high model reliability. Although SMOTE rebalancing improved recall for survived mutants, balancing sensitivity and precision remains challenging. The findings advocate for the development of more balanced datasets in future PMT research to enhance both accuracy and robustness in real-world testing scenarios.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An Analysis of Class Imbalance Challenges in Predictive Mutation Testing

  • Sara Balderas-Díaz,
  • Gabriel Guerrero-Contreras,
  • Pedro Delgado-Pérez,
  • Inmaculada Medina-Bulo

摘要

This study evaluates the efficacy of Predictive Mutation Testing (PMT) models, focusing on the impact of preprocessing, class rebalancing, and model selection on predictive accuracy. By filtering unreached mutants, artificially inflated metrics caused by trivial correlations were minimized, enhancing the ability of the model to identify complex patterns. While ensemble models like CatBoost and XGBoost showed high effectiveness in detecting killed mutants, their performance in identifying survived mutants was limited, underscoring the need for refined strategies to address class imbalance. The study further identifies that previous PMT research often lacks class-specific metrics, such as the Matthews Correlation Coefficient, essential for assessing balanced performance. This oversight can yield misleading evaluations by favoring the majority class, which may falsely indicate high model reliability. Although SMOTE rebalancing improved recall for survived mutants, balancing sensitivity and precision remains challenging. The findings advocate for the development of more balanced datasets in future PMT research to enhance both accuracy and robustness in real-world testing scenarios.