The main purpose of the study is to conduct performance analysis of classification algorithms based on different configuration settings used for software defect prediction and also to find the most appropriate algorithm for software defect prediction and software quality assurance when applied on dataset CM1. In order to measure the performance of classifiers Weka tool was being used for setting up experimental environment. The cross-validation settings 5-fold, 15-fold, 25-fold, and 35-fold was being configured to analyze. Based on the research objectives null hypothesis H01 to H06 were being framed and tested using the one-sample t-test at 5% level of significance. The findings confirm that there is significant difference on performance measure accuracy, Kappa statistics, mean absolute error, precision, recall, and F-measure while comparing classification algorithms based on type of cross-validation being applied in software defect prediction.