A transformer-based stacked ensemble framework for software defect prediction
摘要
Software defect datasets frequently exhibit significant class imbalance, impeding the effectiveness of traditional prediction models. In our benchmark datasets, the ratio of defective to non-defective modules ranges roughly from 1:4 to 1:11, which severely skews classifier learning. To mitigate this issue, we employ the SMOTETomek hybrid resampling method to obtain an approximately 1:1 balanced training distribution. Building upon this balanced representation, we further design a two-tier ensemble framework that integrates classical machine learning classifiers with a transformer-based meta-learner. Unlike existing ensemble approaches that employ static or fixed-weight aggregation, the proposed framework introduces a transformer-based meta-learner that utilizes dynamic self-attention to adaptively combine base classifier outputs for each software module. To the best of our knowledge, this is the first ensemble framework in software defect prediction to replace static aggregation with dynamic attention-based fusion. diverse base classifier outputs using self-attention mechanisms, enabling the model to learn complex inter-model dependencies. Experiments conducted on eight benchmark datasets from the NASA MDP repositories demonstrate that our framework achieves superior performance over state-of-the-art methods. Specifically, the proposed model achieves up to 19.6% improvement in AUC, 25.2% in F1-score, and 22.6% in MCC, compared to leading ensemble-based approaches such as DE-SDP and SDAEsTSE. These results highlight the robustness and generalization capabilities of our architecture in handling class-imbalanced software defect prediction tasks.