AnImaLS: Annotation of Images in Large Scale: what can machines and specialists learn from interaction?

Abstract

In many areas of the Sciences and Engineering, images provide important information about real problems and the assignment by a machine of one or multiple labels per image (annotation) leads to a decision about the problem. However, the machine learning process requires the manual isolation and identification (label assignment) of the content of interest, named sample, in training images. As unlabeled images and samples per image grow large in number, their manual annotation becomes infeasible, leading us to the following research questions: What is the most intuitive way specialists can teach machines to annotate images? What are the tasks and challenges involved in this process? How to minimize human effort with maximum efficacy in machine learning? What can machines and specialists learn from their interaction? This thematic project aims at finding answers to these questions by the study and development of methods for image annotation in large scale, and the construction of decision-making systems (with no user participation) and decision-support systems, wheremachines and specialists cooperate with and learn from each other continuously. The methodology exploits the complementary skills of humans, better in knowledge abstraction, and machines, superior in large scale data processing, in a repetitive way and free of fatigue. The project divides the problem into two steps: sample extraction and interactive machine learning, in view of building a sequence of operations for automated annotation of new images or annotation with minimum user intervention. The study searches to combine image features learned from the data with features that derive from the knowledge of the specialists about the application. The project still aims at technological and scientific innovations for the diagnosis of gastrintestinal parasites in humans and vertebrate animals, by the automated acquisition and annotation of optical microscopy images, and to demonstrate the extension of the methods to other application domains, the project includes studies of medical image annotation using multi-object shape models and interactive annotation of remote sensing images. (AU)