Study and evaluation of mutation test parameterization alternatives aiming at its use in software systems production environments

Abstract

Considering the relevance of automating testing activities for the software production industry and the relevance of mutation-based testing based on scientific evidence, in the regular research project (FAPESP nº 2019/23160-0), entitled "Software Testing Based on Mutation with High Efficiency and Low Technical Debt: Automated Process and Prototype of Free Support Environment" seeks to develop a prototype environment to support mutation testing that is 100% automated, or that is, do not require human interaction or minimize the need for human intervention. Objective: This BPE project aims to carry out a series of experimental studies, considering real software products (proprietary or open source), which allows demonstrating the feasibility of conducting the mutation test in this context, what the results obtained, and how the prototype can support quality assurance in software systems production environments. These experimental studies are a subset of the activities under execution in the regular research project (FAPESP nº 2019/23160-0). Methods: Experimental studies allow thecollection of metrics about the product under test and data from the execution of the mutation test considering different alternatives to reduce the application cost, aiming to demonstrate which are the best alternatives for using this test criterion in real software products, considering time and cost constraints imposed on projects. Expected Results: With this, it is expected the definition of basic guidelines for the use of mutation testing in real software development environments in an automated way, considering the time and cost restrictions imposed on the project. Note that such guidelines should vary based on metrics collected for the software products under testing. Conclusions: The configuration of the prototype to support the application of the mutation test in a 100% automated way demands alternatives for parameterization of the mutation test in order to prioritize what to test of the product under test and which strategies to use for mutation testing should be used to obtain the best cost/benefit ratio. Due to the proven effectiveness of mutation testing in the detection of defects, the use of the technique in real software development environments is already a reality, but still little explored on a large scale, and, with these studies and prototypes, it is expected to provide subsidies for that more companies can use mutation testing in their quality assurance processes. Furthermore, as a result of the period abroad, linked to this BPE project, it is expected to intensificate the cooperation between the research group of Profa. Dr. Ana Paiva and the proponent's research group. (AU)