Software Defect Detection Using Machine Learning on Data from Open-Source Programs

To identify software defects, this paper looks at an effective machine learning (ML) approach. It focuses on distinguishing between software files that are defective and those that are not, and it also explores the idea of employing only structural code metrics to achieve that. To categorize files as defective or non-defective, two real-life datasets were used: one from the source code of twenty-six releases of Notepad++ and one from the source code of twenty-six releases of JuCi++ created using the LocMetrics tool. Using accuracy, precision, recall, F1-score, and their macro/weighted metrics, seven ML models—Decision Tree (DT), Support Vector Machine (SVM), Logistic Regression (LR), Naïve Bayes (NB), Random Forest (RF), and Extra Trees (ET)—were deployed and benchmarked. Randomized Grid Search (RGS) was utilized to optimize each model, followed by 10-fold Cross Validation (CV) for training. The models were assessed twice: first using test data and again using unseen data. The ET model surpassed the other models with an accuracy of 82.09% and continued to dominate the validation and evaluation processes, according to the results from the Notepad++ dataset. On the other hand, the results from the JuCi++ dataset showed that the LR model surpassed others in validation and evaluation procedures. Last, the most important predictors for Notepad++ were the type of source codes (presence of C++), BLOC, and SLOC-L, while for JuCi++ were the type of the file (hpp and cpp extension) and HCWORD, illustrating that the programming language, the clarity, and readability of the code (denoted by the number of blank lines), the logical source lines of code and the header files and their header comments play crucial roles in determining the likelihood of software defects.