Using concepts of content-based image retrieval to implement graphical testing oracles

Automation of testing is an essential requirement to render it viable for software development. Although there are several testing techniques and criteria in many different domains, developing methods to test programs with complex outputs remains an unsolved challenge. This setting includes programs with graphical output, which produce images or interface windows. One possible approach towards automating the testing activity is the use of automatic oracles in which a reference image, taken as correct, can be used to establish a correctness measure in the tested program execution. A method that uses concepts of content-based image retrieval to facilitate oracle automation in the domain of programs with graphics output is presented. Two case studies, one using a computer-aided diagnostic system and one using a Web application, are presented, including some reflections and discussions that demonstrate the feasibility of the proposed approach. Copyright (c) 2011 John Wiley \& Sons, Ltd. (AU)