Advanced search
Start date
(Reference retrieved automatically from Web of Science through information on FAPESP grant and its corresponding number as mentioned in the publication by the authors.)

Compact descriptors for sketch-based image retrieval using a triplet loss convolutional neural network

Full text
Bui, T. [1] ; Ribeiro, L. [2] ; Ponti, M. [2] ; Collomosse, J. [1]
Total Authors: 4
[1] Univ Surrey, CVSSP, Guildford GU2 7XH, Surrey - England
[2] Univ Sao Paulo, Inst Math & Comp Sci ICMC, BR-13566590 Sao Carlos, SP - Brazil
Total Affiliations: 2
Document type: Journal article
Source: COMPUTER VISION AND IMAGE UNDERSTANDING; v. 164, n. SI, p. 27-37, NOV 2017.
Web of Science Citations: 10

\textbackslash{}We present an efficient representation for sketch based image retrieval (SBIR) derived from a triplet loss convolutional neural network (CNN). We treat SBIR as a cross-domain modelling problem, in which a depiction invariant embedding of sketch and photo data is learned by regression over a siamese CNN architecture with half-shared weights and modified triplet loss function. Uniquely, we demonstrate the ability of our learned image descriptor to generalise beyond the categories of object present in our training data, forming a basis for general cross-category SBIR. We explore appropriate strategies for training, and for deriving a compact image descriptor from the learned representation suitable for indexing data on resource constrained e.g. mobile devices. We show the learned descriptors to outperform state of the art SBIR on the defacto standard Flickrl5k dataset using a significantly more compact (56 bits per image, i. e. approximate to 105KB total) search index than previous methods. Datasets and models are available from the CVSSP datasets server at (C) 2017 Elsevier Inc. All rights reserved. (AU)

FAPESP's process: 15/26050-0 - Multiple-instance image ranking for sketch-based image retrieval
Grantee:Leo Sampaio Ferraz Ribeiro
Support Opportunities: Scholarships abroad - Research Internship - Scientific Initiation
FAPESP's process: 16/16111-4 - Feature learning applied to sketch-based image retrieval and low-altitude remote sensing
Grantee:Moacir Antonelli Ponti
Support Opportunities: Regular Research Grants