Construction and evolution of knowledge base in software testing emphasizing automation and experimentation

Abstract

Software Engineering is an area in constant evolution and different methods, techniques, criteria and tools are developed constantly aiming to improve the way software products are designed, contributing to improving the quality of software products. Specifically, regarding software testing, agile methodologies with emphasis on unit testing and pair programming, test driven development prioritizing the creation of tests before coding, and the increasing in the software process maturity level of software companies make the interest of improvements in testing processes gradually increase and high interest. However, such developments and interest may be frustrated if wrong decisions are taken when deciding which techniques, criteria and tools should be employed in a testing process for a given context. Although it is believed that a significant amount of options are available, the decision on which of these numerous techniques, criteria and testing tools to use or even how to combine them in a strategic and appropriated way for specific scenarios cannot be taken randomly based on beliefs or experience from a small group of professionals. Such a decision should be guided scientific evidences and on qualitative and quantitative information demonstrating the benefits of each technique, criteria and tools alone or combined. Moreover, especially in the scenario of micro and small businesses, there is difficulty fitting an innovation process or searching for improvements to the development process, so that innovations and/or improvements end up being postponed or even abandoned. The "Knowledge and Improvement on Test Framework (KITest)" has been proposed aimed at creating a knowledge map about deploying test processes in a generic test process, which can be instantiated to different contexts and scenarios. One point to be further investigated in the KITest Framework is the issue of automation of the process and the form of evolution of knowledge linked to KITest. In this context, this research project aims to investigate and propose a methodology for selecting automated tools to support the testing process based on the structure of the KITest framework and a strategy for its constant evolution based on cooperation and experimentation among specialists and practitioners. This methodology involves defining a taxonomy for test tools, criteria for the selection of testing tools, conducting experimental studies for the identification of adequate test tools for each testing phase, and the instantiation of a process of automated testing based on the identified set of tools. Finally, to enable cooperation among specialists, the establishment of a mechanism for recommendation/approval of artifacts related areas mapped by the KITest framework will be pursued. (AU)