Advanced search
Start date
Betweenand

Learning Visual Clues of the Passage of Time

Grant number: 19/15822-2
Support type:Scholarships abroad - Research Internship - Doctorate
Effective date (Start): December 01, 2019
Effective date (End): November 30, 2020
Field of knowledge:Physical Sciences and Mathematics - Computer Science
Principal researcher:Anderson de Rezende Rocha
Grantee:Rafael Soares Padilha
Supervisor abroad: Nathan Jacobs
Home Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil
Research place: University of Kentucky (UK), United States  
Associated to the scholarship:17/21957-2 - Learning Visual Clues of the Passage of Time, BP.DR

Abstract

As smartphones became ubiquitous in modern life, people started to register and share moments of their lives in social networks. Modern events, from music concerts to a historical building consumed by fire, generate a massive amount of data, depicting different points of view and moments in time. To understand how the event unfolded, it is essential to organize available footage in time and reconstruct the order of facts. However, data produced in real-world events not always have reliable annotations with respect to time-of-capture, hindering manual reconstruction of its timeline.In this internship project, we propose methods to automatically sort images and videos in time by analyzing their visual content. For this, we will train an event-agnostic model to learn directly from data a representation that emphasizes the passage of time. Then, this representation will be used as input to a model that infers the chronological order between a pair of images. Finally, considering pairwise relations between available data, we will reconstruct the timeline of that event. The proposed methods will be optimized in training setups that consider different levels of annotation with respect to time-of-capture, and will be evaluated with data gathered from real-world events.

News published in Agência FAPESP Newsletter about the scholarship:
Articles published in other media outlets (0 total):
More itemsLess items
VEICULO: TITULO (DATA)
VEICULO: TITULO (DATA)

Please report errors in scientific publications list by writing to: cdi@fapesp.br.