Can we spot energy regressions using developers tests?

Context

Software Energy Consumption is gaining more and more attention. In this paper, we tackle the problem of warning developers about the increase of SEC of their programs during Continuous Integration (CI).

Objective

In this study, we investigate if the CI can leverage developers’ tests to perform energy regression testing. Energy regression is similar to performance regression but focuses on the energy consumption of the program instead of standard performance indicators, like execution time or memory consumption.

Method

We perform an exploratory study of the usage of developers’ tests for energy regression testing. We first investigate if developers’ tests can be used to obtain stable SEC indicators. Then, we evaluate if comparing the SEC of developers’ tests between two versions can pinpoint energy regressions introduced by automated program mutations. Finally, we manually evaluate several real commits pinpointed by our approach.

Results

Our study will pave the way for automated SEC regression tools that can be readily deployed inside an existing CI infrastructure to raise awareness of SEC issues among practitioners.