Scholarlink

<p>Previous studies have indicated that the performance of Just-In-Time Software Defect Prediction (JIT-SDP) models can degrade over time due to Concept Drift (CD), which refers to changes in the characteristics of training data of evolving software, such as code modifications and environmental changes. In the literature, baseline methods have been applied for CD Detection (CDD) through the detection of significant changes in the Error Rate (ER) of an incremental learning model over time, relying on labeled test data. This dependency creates delays in identifying model performance instability. Nevertheless, such methods can serve as baseline approaches for evaluating new models. In this paper, we propose a novel approach, Concept Drift Detection via Model Interpretation (CDD_MI), to predict significant instabilities in JIT-SDP models over time using monitoring significant changes in MI vectors. MI highlights the importance of features in predicting class outcomes without requiring labeled test data. Our study focuses on identifying significant changes in the explanation of incremental and non-incremental models over time, specifically analyzing positive, negative, and average effect sizes of MI. This research offers four main contributions: 1- We analyze the inconsistency of MI over time in evolving JIT-SDP models. 2- Our method eliminates the need for labeled test datasets for CD detection. 3- We identify features that significantly influence CD. 4- A key challenge in JIT-SDP studies is the absence of labeled CD data. To address this limitation, we compare our method with the CDD method based on threshold-dependent and threshold-independent performance measures. In this paper, for the first time, the CDD method based on the monitoring of various non-ER performance measures has been used to detect and evaluate discovered CDs. Our study evaluates the proposed method on 20 well-established Java datasets, ensuring consistency with prior research in this domain. Additionally, to assess the generalizability of our approach, we extended our evaluation to two non-Java datasets. The results across all datasets—including both Java and non-Java—indicate that the CD points of the JIT-SDP model can be predicted over time with a high degree of accuracy without requiring a dataset of software defects labeled with CD points, reinforcing the robustness of our methodology.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Detecting concept drift in just-in-time software defect prediction using model interpretation

  • 期刊论文

  • 发表日期：

    2026年2月1日

查看全文
  • Zeynab Chitsazian,
  • Saeed Sedighian Kashi,
  • Amin Nikanjam

摘要

Previous studies have indicated that the performance of Just-In-Time Software Defect Prediction (JIT-SDP) models can degrade over time due to Concept Drift (CD), which refers to changes in the characteristics of training data of evolving software, such as code modifications and environmental changes. In the literature, baseline methods have been applied for CD Detection (CDD) through the detection of significant changes in the Error Rate (ER) of an incremental learning model over time, relying on labeled test data. This dependency creates delays in identifying model performance instability. Nevertheless, such methods can serve as baseline approaches for evaluating new models. In this paper, we propose a novel approach, Concept Drift Detection via Model Interpretation (CDD_MI), to predict significant instabilities in JIT-SDP models over time using monitoring significant changes in MI vectors. MI highlights the importance of features in predicting class outcomes without requiring labeled test data. Our study focuses on identifying significant changes in the explanation of incremental and non-incremental models over time, specifically analyzing positive, negative, and average effect sizes of MI. This research offers four main contributions: 1- We analyze the inconsistency of MI over time in evolving JIT-SDP models. 2- Our method eliminates the need for labeled test datasets for CD detection. 3- We identify features that significantly influence CD. 4- A key challenge in JIT-SDP studies is the absence of labeled CD data. To address this limitation, we compare our method with the CDD method based on threshold-dependent and threshold-independent performance measures. In this paper, for the first time, the CDD method based on the monitoring of various non-ER performance measures has been used to detect and evaluate discovered CDs. Our study evaluates the proposed method on 20 well-established Java datasets, ensuring consistency with prior research in this domain. Additionally, to assess the generalizability of our approach, we extended our evaluation to two non-Java datasets. The results across all datasets—including both Java and non-Java—indicate that the CD points of the JIT-SDP model can be predicted over time with a high degree of accuracy without requiring a dataset of software defects labeled with CD points, reinforcing the robustness of our methodology.