Classification algorithms are used in fault prediction domain to accurately predict the fault-proneness of a class based on the characteristics of the software system. They represent a beneficial tool in the decision-making, as they enable the software developers and testers to proficiently distribute the software development resources. The data used by the classification algorithms are class imbalanced as faults usually lie in the small number of software classes. The classification algorithms applied to class imbalanced data tend to produce sub-optimal dichotomous ability. The class imbalanced problem can be reduced by using sampling techniques capable of balancing the class distribution. The objective of this study is to evaluate the efficiency of sampling method in improving the classification performance of the threshold techniques with an emphasis on how such improvements can contribute to more efficient and sustainable marketing strategies. In this study, the authors applied Synthetic Minority Oversampling Technique (SMOTE) for the re-sampling purpose. The authors selected two threshold techniques, Cohen’s kappa and odds ratio, to compute the optimal cut-off values of Chidamber and Kemerer metric suite. The optimal cut-off or threshold value can efficiently discriminate the faulty and non-faulty classes. The threshold techniques were applied on the original dataset and re-sampled dataset in order to verify the performance difference of the selected threshold techniques. The Wilcoxon signed rank test result showed that the prediction performance of the kappa method on re-sampled dataset is significantly better than the performance on the original dataset, which could directly benefit green marketing strategies by enhancing predictive accuracy. The authors also compared the outcome of selected threshold techniques with ROC curve and found that the joint use of Kappa and SMOTE can produce the classification performance as good as the ROC method.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Performance of Threshold Techniques on Imbalanced Fault Dataset: The Effect of SMOTE Method

  • Guneet Singh Bhalla,
  • Pardeep Bawa Sharma,
  • Navneet Kaur

摘要

Classification algorithms are used in fault prediction domain to accurately predict the fault-proneness of a class based on the characteristics of the software system. They represent a beneficial tool in the decision-making, as they enable the software developers and testers to proficiently distribute the software development resources. The data used by the classification algorithms are class imbalanced as faults usually lie in the small number of software classes. The classification algorithms applied to class imbalanced data tend to produce sub-optimal dichotomous ability. The class imbalanced problem can be reduced by using sampling techniques capable of balancing the class distribution. The objective of this study is to evaluate the efficiency of sampling method in improving the classification performance of the threshold techniques with an emphasis on how such improvements can contribute to more efficient and sustainable marketing strategies. In this study, the authors applied Synthetic Minority Oversampling Technique (SMOTE) for the re-sampling purpose. The authors selected two threshold techniques, Cohen’s kappa and odds ratio, to compute the optimal cut-off values of Chidamber and Kemerer metric suite. The optimal cut-off or threshold value can efficiently discriminate the faulty and non-faulty classes. The threshold techniques were applied on the original dataset and re-sampled dataset in order to verify the performance difference of the selected threshold techniques. The Wilcoxon signed rank test result showed that the prediction performance of the kappa method on re-sampled dataset is significantly better than the performance on the original dataset, which could directly benefit green marketing strategies by enhancing predictive accuracy. The authors also compared the outcome of selected threshold techniques with ROC curve and found that the joint use of Kappa and SMOTE can produce the classification performance as good as the ROC method.