Mutation-based software testing with high efficiency and low technical debt: automated process and free support environment prototype

Abstract

Considering the relevance of automating testing activities for the software product industry and the relevance of mutation-based testing from scientific evidence, this project aims to define a mutation-based testing process that can be performed in a 100% automated manner through a supportive test environment, specified, and validated with industrial and open-source applications. Methods: Investigate and automate three classic mutation test problems: 1) mutant generation; 2) execution of mutants, and 3) analysis of live and equivalent mutants in the test process. In the generation, it is intended to use static analysis and control and data flow information to select the points that the mutations shall be performed, in addition to selecting specific types of operators to be used depending on the characteristics of the product under test. In the execution of mutants, the selection of good test cases with a high probability of killing non-equivalent mutants is very important. To do this, it is known that different automatic test data generators must be combined and, possibly, new generation algorithms must be developed to kill mutants generated by specific mutation operators. In the analysis of living and equivalent mutants, we need to define and evaluate automated strategies for the determination of equivalent mutants, using heuristics, Bayesian learning, and the frequency of execution of the mutants by the test cases. Expected results: a free testing process and a support test environment, as well as experimentation data that allows the generation of benchmarks for testing C, Java, and Python, as well as for the development of new research related to the mutation test. Process and Environment will be applied and evaluated in industrial partners which are formally supporting the present project with interest in its results to be used on production. Technical debt in the context of this proposal will be conducted to characterize the risks associated with the production and release of software products based on mutation testing criterion. Conclusions: It is intended to enable the application of mutation testing as an important mechanism for ensuring the quality of software products in a fully automated way, favoring the technological transfer and application of the mutation test to the industry, with the consequent evolution of the software production capacity of the Brazilian software industry. (AU)