Scholarlink

<p>Performance is a critical quality attribute in software development, yet the impact of method-level code changes on performance evolution remains poorly understood. While developers often make intuitive assumptions about which types of modifications are likely to cause performance regressions or improvements, these beliefs lack empirical validation at a fine-grained level. We conducted a large-scale empirical study analyzing performance evolution in 15 mature open-source Java projects hosted on GitHub. Our analysis encompassed 739 commits containing 1,499 method-level code changes, using Java Microbenchmark Harness (JMH) to benchmark the performance of these methods before and after the changes, thereafter measuring the significance and magnitude of the performance variations. We employed bytecode instrumentation to capture method-specific execution metrics and systematically analyzed four key aspects: the performance change patterns, the impact of code change types, the developer and complexity factors, as well as the impact of project domain and size. Our findings reveal that 32.7% of method-level changes result in measurable performance impacts, with regressions occurring 1.3 times more frequently than improvements (18.5% vs 14.2%). We also observe that certain code change types, such as <i>algorithmic changes</i>, demonstrate a higher chance of performance improvement, but also carry substantial regression risks. While senior developers produce more stable changes, junior developers are more likely to be associated with aggressive performance changes; besides, we observe an increasing trend of performance regressions associated with more complex code. Finally, our findings show that different project domains and scales exhibit distinct performance characteristics, which suggest the need for domain-specific performance management strategies that account for the project scales. Our study provides empirical insights for integrating automated performance testing and monitoring into continuous integration and deployment pipelines.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An empirical study on method-level performance evolution in open-source Java projects

  • 期刊论文

  • 发表日期：

    2026年8月4日

查看全文
  • Kaveh Shahedi,
  • Nana Gyambrah,
  • Heng Li,
  • Maxime Lamothe,
  • Foutse Khomh

摘要

Performance is a critical quality attribute in software development, yet the impact of method-level code changes on performance evolution remains poorly understood. While developers often make intuitive assumptions about which types of modifications are likely to cause performance regressions or improvements, these beliefs lack empirical validation at a fine-grained level. We conducted a large-scale empirical study analyzing performance evolution in 15 mature open-source Java projects hosted on GitHub. Our analysis encompassed 739 commits containing 1,499 method-level code changes, using Java Microbenchmark Harness (JMH) to benchmark the performance of these methods before and after the changes, thereafter measuring the significance and magnitude of the performance variations. We employed bytecode instrumentation to capture method-specific execution metrics and systematically analyzed four key aspects: the performance change patterns, the impact of code change types, the developer and complexity factors, as well as the impact of project domain and size. Our findings reveal that 32.7% of method-level changes result in measurable performance impacts, with regressions occurring 1.3 times more frequently than improvements (18.5% vs 14.2%). We also observe that certain code change types, such as algorithmic changes, demonstrate a higher chance of performance improvement, but also carry substantial regression risks. While senior developers produce more stable changes, junior developers are more likely to be associated with aggressive performance changes; besides, we observe an increasing trend of performance regressions associated with more complex code. Finally, our findings show that different project domains and scales exhibit distinct performance characteristics, which suggest the need for domain-specific performance management strategies that account for the project scales. Our study provides empirical insights for integrating automated performance testing and monitoring into continuous integration and deployment pipelines.