Assessing model-driven mutation testing of Java bytecode
摘要
Mutation testing is an approach to checking the robustness of test suites. The program code is slightly modified by mutations to inject bugs, and a test suite is robust enough if it finds them. Mutation testing tools provide sets of mutation operators, such as swapping arithmetic operators, to make small modifications to the program. The results of mutation tests depend directly on the possible mutations. These mutations should cause actual changes in the program behavior, but also should not prevent the program from being loaded and executed. The more advanced mutations are, the more they challenge the test suite. Existing non-model-based mutation testing tools do not support the definition of advanced mutation operators that go beyond manipulating a small number of adjacent instructions within a single method. Thus, we present a model-driven approach where mutations of Java bytecode can be flexibly defined as model transformations. Our tool, Model-based Mutation Testing (MMT), implements this approach and includes model transformations for conventional and advanced mutation operators, such as deleting overridden methods or changing type casts. To evaluate the effectiveness and efficiency of model-driven mutation testing, we have applied MMT to all projects and versions in Defects4J, a well-established collection of real-world Java projects with reproducible bugs. We check for MMT’s ability to generate mutants close to real bugs and compare it with the non-model-based mutation testing tools Jumble, PIT, and