Multi-label annotation of natural image databases with minimal supervision

Abstract

The growth of video and image databases has made automatic image annotation very important for content-based image retrieval. Content-based retrieval is a current research topic, and has been used for retrieval of several image modalities (e.g., photography, videos, tomographies, remote sensing images). The objective of this project is to study and develop techniques for automatic annotation of natural images from a database -- i.e., associating a label (keyword) to each object of interest in a given image. A promising methodology consists on automatically segmenting the image, extracting features from its regions and annotating each region using a pattern classifier. There are several challenges to be investigated in each of those steps: segmentation techniques, learning feature descriptors (deep learning, convolutional networks), active learning applied to pattern classifiers, and post-processing techniques to keep consistent image annotations between connected regions. We are particularly interested in selecting training samples (regions) using active learning techniques, intending to enhance the accuracy of the annotations and minimize the effort employed by the user to annotate the training set of the classifier. We will also investigate techniques for learning region descriptors for natural images and segmentation post-processing. The results will be validated using image datasets for testing already established in the literature. (AU)