An extensible infrastructure for planning, running, analyzing and packaging experiments in software engineering

Abstract

One of the main objectives of Empirical Software Engineering is the improvement of methods, techniques, and tools for Software Engineering (SE) through the conduction and execution of experiments. Although research in empirical software engineering is well established in the literature, it is possible to verify that there are few supporting environments for conducting experiments, despite the increasing demand for this kind of study within the SE community. Moreover, existing approaches lack a number of important features. For instance, they do not offer extensible mechanisms for all required phases of experimentation in the different domains of SE, integration with existing software development environments, ways of how to define metrics to be collected, automatic organization mechanisms to analyze collected data using a specific statistical analyzes method, and remote execution of experiments. In this way, the primary objective of this research is to develop a computational infrastructure for the stages of the experimentation process, supporting extension and customization for the types of empirical SE studies, into the domain of Software Testing. Furthermore, such an infrastructure shall provide innovation and improvements concerning the proposed approaches in the literature. In order to show the feasibility of the proposed infrastructure, we plan to conduct several validation steps, including the application of User eXperience (UX) and Developer eXperience (DX). As part of the assessment of the approach, we will also reproduce previous experimental studies carried out by our research group to see whether reproductions of experiments is feasible through the proposed computational infrastructure. (AU)