Scholarlink

<p>Software defect prediction historically relies on code and history metrics, largely ignoring developer-level indicators. Yet, because defects typically stem from human error, human-factors theory offers complementary, theoretically grounded predictors. This paper introduces a framework for deriving prediction metrics from human-factors theory and instantiates it with two metrics, <i>alertness</i> and <i>memory_decay</i>, operationalizing developer cognitive state during code editing. We evaluate these metrics on twenty-one large-scale open-source projects under two complementary protocols: a discovery-oriented (in-sample) evaluation and a time-aware generalization evaluation on a subset of eighteen projects. Under discovery evaluation, models using cognitive metrics dominate state-of-the-art baselines on every project (PR-AUC 0.64 vs. 0.43), and have surfaced over thirty real, previously-unreported defects across nine critical infrastructure projects, including two assigned CVEs and accepted upstream fixes in systems such as NASA Trick. Under time-aware cross-project evaluation, cognitive metrics encode the strongest cross-project predictive signal in the benchmark: <i>alertness</i> ranks first by mean SHAP importance and <i>memory_decay</i> exhibits the lowest rank volatility, a stability independently confirmed by chance-corrected selection-stability analysis; the configuration combining cognitive and traditional metrics significantly outperforms the traditional baseline alone (Holm-corrected <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(p = 0.018\)</EquationSource> </InlineEquation>, large effect). Furthermore, cognitive configurations demonstrate a significant, large-effect reverse generalization gap—performing better under cross-project transfer than within-project evaluation—while substituting traditional edit-frequency counts to produce calibration-neutral models. Consequently, our findings validate cognitive theory for explainable, actionable, and interpretable safety-critical defect prediction, laying empirical groundwork to evaluate analogous issues in LLM-generated code through the behavioral study of AI.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Actionable insights from developer behavior: a practical approach to software defect prediction

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月23日

查看全文
  • Carlos Andrés Ramírez Cataño,
  • Makoto Itoh

摘要

Software defect prediction historically relies on code and history metrics, largely ignoring developer-level indicators. Yet, because defects typically stem from human error, human-factors theory offers complementary, theoretically grounded predictors. This paper introduces a framework for deriving prediction metrics from human-factors theory and instantiates it with two metrics, alertness and memory_decay, operationalizing developer cognitive state during code editing. We evaluate these metrics on twenty-one large-scale open-source projects under two complementary protocols: a discovery-oriented (in-sample) evaluation and a time-aware generalization evaluation on a subset of eighteen projects. Under discovery evaluation, models using cognitive metrics dominate state-of-the-art baselines on every project (PR-AUC 0.64 vs. 0.43), and have surfaced over thirty real, previously-unreported defects across nine critical infrastructure projects, including two assigned CVEs and accepted upstream fixes in systems such as NASA Trick. Under time-aware cross-project evaluation, cognitive metrics encode the strongest cross-project predictive signal in the benchmark: alertness ranks first by mean SHAP importance and memory_decay exhibits the lowest rank volatility, a stability independently confirmed by chance-corrected selection-stability analysis; the configuration combining cognitive and traditional metrics significantly outperforms the traditional baseline alone (Holm-corrected \(p = 0.018\) , large effect). Furthermore, cognitive configurations demonstrate a significant, large-effect reverse generalization gap—performing better under cross-project transfer than within-project evaluation—while substituting traditional edit-frequency counts to produce calibration-neutral models. Consequently, our findings validate cognitive theory for explainable, actionable, and interpretable safety-critical defect prediction, laying empirical groundwork to evaluate analogous issues in LLM-generated code through the behavioral study of AI.