Advanced search
Start date

Weakly supervised learning for compressed video analysis on retrieval and classification tasks for visual alert

Grant number: 17/25908-6
Support Opportunities:Research Grants - Research Partnership for Technological Innovation - PITE
Duration: February 01, 2019 - January 31, 2023
Field of knowledge:Physical Sciences and Mathematics - Computer Science
Convênio/Acordo: Microsoft Research
Principal Investigator:João Paulo Papa
Grantee:João Paulo Papa
Host Institution: Instituto de Geociências e Ciências Exatas (IGCE). Universidade Estadual Paulista (UNESP). Campus de Rio Claro. Rio Claro , SP, Brazil
Host Company: Microsoft Informática Ltda
City: São PauloRio Claro
Pesquisadores principais:
( Últimos )
Daniel Carlos Guimarães Pedronette ; Fabio Augusto Faria ; Jurandy Gomes de Almeida Junior
Pesquisadores principais:
( Antigos )
João Paulo Papa
Associated researchers:João Paulo Papa
Associated scholarship(s):22/01246-2 - A comparative analysis of depth feature for multimedia recognition tasks, BP.IC
21/10048-7 - Support for computational environments and experiments execution: data acquisition, categorization, and maintenance, BP.TT
21/10547-3 - Investigation of multi-level representations in multimedia recognition tasks, BP.IC
+ associated scholarships 21/02739-0 - Visual attention models based on compressed domain video analysis techniques, BP.MS
21/01870-5 - Multi-level Representation Fusion Methods based on Weakly Supervised Learning, BP.MS
21/02023-4 - Classifier Selection Strategies based on Genetic Programming for Multimedia Recognition, BP.MS
20/11366-0 - Support for computational environments and experiments execution: weakly-supervised and classification fusion methods, BP.TT
20/12101-0 - Support for computational environments and experiments execution: data acquisition, categorization and maintenance, BP.TT
20/08770-3 - Open set methods based on deep networks for multimedia recognition, BP.MS
20/08854-2 - Investigation of graph-based contextual measures for weakly-supervised learning, BP.IC
19/11104-8 - A comparative analysis of rank correlation measures for weakly-supervised learning, BP.IC
19/15837-0 - Restricted Boltzmann Machines applied to video-based action recognition, BP.IC
19/10998-5 - Investigation of compressed domain video features based on deep neural networks, BP.IC
19/07825-1 - Deep Boltzmann machines for event recognition in videos, BP.MS
19/04754-6 - Weakly supervised learning strategies through Rank-based measures, BP.MS - associated scholarships


Several machine learning techniques have relied on large labeled data sets to construct predictive models and solving supervised learning tasks. The use of deep learning techniques can be highlighted, since it have been broadly and successfully used in various domains. On the other hand, in many circumstances, the labeled sets are unavailable or insufficient to train effective supervised models. Such scenarios have been mainly addressed by unsupervised learning techniques, which consider the unlabeled data to learn about its structure. However, the use of completely unsupervised methods still remains a research challenge in many scenarios and situations. A promising solution is based on the use of weakly supervised approaches, capable of performing effective learning tasks based on incomplete or inaccurate labeled sets. In this project, we intend to investigate the analysis, retrieval, and classification of compressed video domain based on small training sets. The main object of the project consists in to investigate and propose methods capable of analyzing compressed video sequences and trigger alerts according to considered applications. Such approaches can be useful and relevant in several domains, ranging from surveillance, medical and industrial environments to smart homes. The fundamental research challenge consists in making use of different techniques in order to analyze, represent, and classification videos using restricted labeled data. The proposed approach aims at exploiting the maximum available information, in order to become the approach suitable for operating with small training datasets. We intend to exploit: (i) deep learning representations; (ii) contextual unsupervised measures and; (iii) fusion techniques, in order to extend the initial labeled sets. The first challenge to be addressed is to analyze and represent videos in the compressed domain using deep learning techniques. Based on such representations, we intend to investigate strategies for expanding the training sets using unsupervised contextual measures. Given the obtained labeled sets, fusion strategies will be used to combined diverse classification methods and triggering alerts. Although the methods which will investigated can be used in several domains, we intend to select domains to validate the proposed approaches. The selection will be performed considering the existence of public available datasets to conduct experimental evaluations. (AU)

Articles published in Agência FAPESP Newsletter about the research grant:
Articles published in other media outlets (0 total):
More itemsLess items

Scientific publications (15)
(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)
GUIMARAES PEDRONETTE, DANIEL CARLOS; VALEM, LUCAS PASCOTTI; ALMEIDA, JURANDY; TONES, RICARDO DA S.. Multimedia Retrieval Through Unsupervised Hypergraph-Based Manifold Ranking. IEEE Transactions on Image Processing, v. 28, n. 12, p. 5824-5838, . (14/50715-9, 16/50250-1, 17/25908-6, 17/20945-0, 14/12236-1, 16/06441-7, 18/15597-6, 13/50155-0, 17/02091-4, 15/24494-8)
GUIMARAES PEDRONETTE, DANIEL CARLOS; WENG, YING; BALDASSIN, ALEXANDRO; HOU, CHAOHUAN. Semi-supervised and active learning through Manifold Reciprocal kNN Graph for image retrieval. Neurocomputing, v. 340, p. 19-31, . (17/25908-6, 13/08645-0)
GUIMARAES PEDRONETTE, DANIEL CARLOS; VALEM, LUCAS PASCOTTI; TORRES, RICARDO DA S.. A BFS-Tree of ranking references for unsupervised manifold learning. PATTERN RECOGNITION, v. 111, . (16/50250-1, 15/24494-8, 13/50155-0, 18/15597-6, 13/50169-1, 17/20945-0, 14/12236-1, 17/25908-6, 14/50715-9)
DE SOUZA, RENATO WILLIAM R.; DE OLIVEIRA, JOAO VITOR CHAVES; PASSOS, JR., LEANDRO A.; DING, WEIPING; PAPA, JOAO P.; DE ALBUQUERQUE, VICTOR HUGO C.. A Novel Approach for Optimum-Path Forest Classification Using Fuzzy Logic. IEEE TRANSACTIONS ON FUZZY SYSTEMS, v. 28, n. 12, p. 3076-3086, . (13/07375-0, 17/25908-6, 18/21934-5, 16/19403-6, 14/12236-1)
CAMPOS, VICTOR DE ABREU; GUIMARAES PEDRONETTE, DANIEL CARLOS. A framework for speaker retrieval and identification through unsupervised learning. COMPUTER SPEECH AND LANGUAGE, v. 58, p. 153-174, . (17/25908-6, 15/07934-4, 18/15597-6)
SANTANA, MARCOS C. S.; PASSOS, JR., LEANDRO APARECIDO; MOREIRA, THIERRY P.; COLOMBO, DANILO; DE ALBUQUERQUE, VICTOR HUGO C.; PAPA, JOAO PAULO. A Novel Siamese-Based Approach for Scene Change Detection With Applications to Obstructed Routes in Hazardous Environments. IEEE INTELLIGENT SYSTEMS, v. 35, n. 1, p. 44-53, . (14/12236-1, 13/07375-0, 17/25908-6, 16/19403-6)
SCARPARO, DANIELE CRISTINA; PINHEIRO SALVADEO, DENIS HENRIQUE; GUIMARAES PEDRONETTE, DANIEL CARLOS; BARUFALDI, BRUNO; ARNOLD MAIDMENT, ANDREW DOUGLAS. Evaluation of denoising digital breast tomosynthesis data in both projection and image domains and a study of noise model on digital breast tomosynthesis image domain. JOURNAL OF MEDICAL IMAGING, v. 6, n. 3, . (16/09714-4, 17/17811-2, 17/25908-6)
RODER, MATEUS; PASSOS, LEANDRO APARECIDO; DE ROSA, GUSTAVO H.; DE ALBUQUERQUE, VICTOR HUGO C.; PAPA, JOAO PAULO. Reinforcing learning in Deep Belief Networks through nature-inspired optimization. APPLIED SOFT COMPUTING, v. 108, . (19/07825-1, 18/21934-5, 17/25908-6, 19/07665-4, 19/02205-5, 13/07375-0, 14/12236-1)
GUIMARAES PEDRONETTE, DANIEL CARLOS; LATECKI, LONGIN JAN. Rank-based self-training for graph convolutional networks. INFORMATION PROCESSING & MANAGEMENT, v. 58, n. 2, . (17/25908-6, 18/15597-6)
DE ROSA, GUSTAVO H.; PAPA, JOAO P.; YANG, XIN-SHE. A nature-inspired feature selection approach based on hypercomplex information. APPLIED SOFT COMPUTING, v. 94, . (13/07375-0, 16/19403-6, 14/12236-1, 17/25908-6, 17/02286-0, 19/02205-5)
VALEM, LUCAS PASCOTTI; GUIMARAES PEDRONETTE, DANIEL CARLOS. Graph -based selective rank fusion for unsupervised image retrieval. PATTERN RECOGNITION LETTERS, v. 135, p. 82-89, . (17/25908-6, 13/08645-0, 17/02091-4, 18/15597-6)
VALEM, LUCAS PASCOTTI; GUIMARAES PEDRONETTE, DANIEL CARLOS. Unsupervised selective rank fusion for image retrieval tasks. Neurocomputing, v. 377, p. 182-199, . (17/25908-6, 17/02091-4, 18/15597-6)
GUIMARAES PEDRONETTE, DANIEL CARLOS; PASCOTTI VALEM, LUCAS; LATECKI, LONGIN JAN. Efficient Rank-Based Diffusion Process with Assured Convergence. JOURNAL OF IMAGING, v. 7, n. 3, . (20/11366-0, 18/15597-6, 17/25908-6)
ROZIN, BIONDA; PEREIRA-FERRERO, VANESSA HELENA; LOPES, LEONARDO TADEU; PEDRONETTE, DANIEL CARLOS GUIMARAES. A rank-based framework through manifold learning for improved clustering tasks. INFORMATION SCIENCES, v. 580, p. 202-220, . (17/25908-6, 20/02183-9, 20/08854-2, 18/15597-6)
SUGI AFONSO, LUIS CLAUDIO; RODRIGUES, DOUGLAS; PAPA, JOAO PAULO. Nature-inspired optimum-path forest. EVOLUTIONARY INTELLIGENCE, . (17/25908-6, 18/21934-5, 14/12236-1, 13/07375-0, 19/07665-4)

Please report errors in scientific publications list by writing to: