Advanced search
Start date

Weakly supervised learning for compressed video analysis on retrieval and classification tasks for visual alert

Grant number: 17/25908-6
Support type:Research Grants - Research Partnership for Technological Innovation - PITE
Duration: February 01, 2019 - January 31, 2023
Field of knowledge:Physical Sciences and Mathematics - Computer Science
Cooperation agreement: Microsoft Research
Principal researcher:João Paulo Papa
Grantee:João Paulo Papa
Home Institution: Instituto de Geociências e Ciências Exatas (IGCE). Universidade Estadual Paulista (UNESP). Campus de Rio Claro. Rio Claro , SP, Brazil
Company: Microsoft Informática Ltda
City: Rio Claro
Pesquisadores principais:
( Atuais )
Daniel Carlos Guimarães Pedronette ; Fabio Augusto Faria ; Jurandy Gomes de Almeida Junior
Pesquisadores principais:
( Anteriores )
João Paulo Papa
Assoc. researchers:João Paulo Papa
Associated scholarship(s):21/10048-7 - Support for computational environments and experiments execution: data acquisition, categorization, and maintenance, BP.TT
21/10547-3 - Investigation of multi-level representations in multimedia recognition tasks, BP.IC
21/02023-4 - Classifier Selection Strategies based on Genetic Programming for Multimedia Recognition, BP.MS
+ associated scholarships 21/02739-0 - Visual attention models based on compressed domain video analysis techniques, BP.MS
21/01870-5 - Multi-level Representation Fusion Methods based on Weakly Supervised Learning, BP.MS
20/11366-0 - Support for computational environments and experiments execution: weakly-supervised and classification fusion methods, BP.TT
20/12101-0 - Support for computational environments and experiments execution: data acquisition, categorization and maintenance, BP.TT
20/08770-3 - Open set methods based on deep networks for multimedia recognition, BP.MS
20/08854-2 - Investigation of graph-based contextual measures for weakly-supervised learning, BP.IC
19/11104-8 - A comparative analysis of rank correlation measures for weakly-supervised learning, BP.IC
19/15837-0 - Restricted Boltzmann Machines applied to video-based action recognition, BP.IC
19/10998-5 - Investigation of compressed domain video features based on deep neural networks, BP.IC
19/07825-1 - Deep Boltzmann machines for event recognition in videos, BP.MS
19/04754-6 - Weakly supervised learning strategies through Rank-based measures, BP.MS - associated scholarships


Several machine learning techniques have relied on large labeled data sets to construct predictive models and solving supervised learning tasks. The use of deep learning techniques can be highlighted, since it have been broadly and successfully used in various domains. On the other hand, in many circumstances, the labeled sets are unavailable or insufficient to train effective supervised models. Such scenarios have been mainly addressed by unsupervised learning techniques, which consider the unlabeled data to learn about its structure. However, the use of completely unsupervised methods still remains a research challenge in many scenarios and situations. A promising solution is based on the use of weakly supervised approaches, capable of performing effective learning tasks based on incomplete or inaccurate labeled sets. In this project, we intend to investigate the analysis, retrieval, and classification of compressed video domain based on small training sets. The main object of the project consists in to investigate and propose methods capable of analyzing compressed video sequences and trigger alerts according to considered applications. Such approaches can be useful and relevant in several domains, ranging from surveillance, medical and industrial environments to smart homes. The fundamental research challenge consists in making use of different techniques in order to analyze, represent, and classification videos using restricted labeled data. The proposed approach aims at exploiting the maximum available information, in order to become the approach suitable for operating with small training datasets. We intend to exploit: (i) deep learning representations; (ii) contextual unsupervised measures and; (iii) fusion techniques, in order to extend the initial labeled sets. The first challenge to be addressed is to analyze and represent videos in the compressed domain using deep learning techniques. Based on such representations, we intend to investigate strategies for expanding the training sets using unsupervised contextual measures. Given the obtained labeled sets, fusion strategies will be used to combined diverse classification methods and triggering alerts. Although the methods which will investigated can be used in several domains, we intend to select domains to validate the proposed approaches. The selection will be performed considering the existence of public available datasets to conduct experimental evaluations. (AU)

Articles published in Agência FAPESP Newsletter about the research grant:
Articles published in other media outlets (0 total):
More itemsLess items

Scientific publications (16)
(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)
ROZIN, BIONDA; PEREIRA-FERRERO, VANESSA HELENA; LOPES, LEONARDO TADEU; PEDRONETTE, DANIEL CARLOS GUIMARAES. A rank-based framework through manifold learning for improved clustering tasks. INFORMATION SCIENCES, v. 580, p. 202-220, NOV 2021. Web of Science Citations: 0.
RODER, MATEUS; PASSOS, LEANDRO APARECIDO; DE ROSA, GUSTAVO H.; DE ALBUQUERQUE, VICTOR HUGO C.; PAPA, JOAO PAULO. Reinforcing learning in Deep Belief Networks through nature-inspired optimization. APPLIED SOFT COMPUTING, v. 108, SEP 2021. Web of Science Citations: 0.
SUGI AFONSO, LUIS CLAUDIO; RODRIGUES, DOUGLAS; PAPA, JOAO PAULO. Nature-inspired optimum-path forest. EVOLUTIONARY INTELLIGENCE, SEP 2021. Web of Science Citations: 0.
GUIMARAES PEDRONETTE, DANIEL CARLOS; PASCOTTI VALEM, LUCAS; LATECKI, LONGIN JAN. Efficient Rank-Based Diffusion Process with Assured Convergence. JOURNAL OF IMAGING, v. 7, n. 3 MAR 2021. Web of Science Citations: 0.
GUIMARAES PEDRONETTE, DANIEL CARLOS; VALEM, LUCAS PASCOTTI; TORRES, RICARDO DA S. A BFS-Tree of ranking references for unsupervised manifold learning. PATTERN RECOGNITION, v. 111, MAR 2021. Web of Science Citations: 0.
GUIMARAES PEDRONETTE, DANIEL CARLOS; LATECKI, LONGIN JAN. Rank-based self-training for graph convolutional networks. INFORMATION PROCESSING & MANAGEMENT, v. 58, n. 2 MAR 2021. Web of Science Citations: 0.
DE SOUZA, RENATO WILLIAM R.; DE OLIVEIRA, JOAO VITOR CHAVES; PASSOS, JR., LEANDRO A.; DING, WEIPING; PAPA, JOAO P.; DE ALBUQUERQUE, VICTOR HUGO C. A Novel Approach for Optimum-Path Forest Classification Using Fuzzy Logic. IEEE TRANSACTIONS ON FUZZY SYSTEMS, v. 28, n. 12, p. 3076-3086, DEC 2020. Web of Science Citations: 5.
DE ROSA, GUSTAVO H.; PAPA, JOAO P.; YANG, XIN-SHE. A nature-inspired feature selection approach based on hypercomplex information. APPLIED SOFT COMPUTING, v. 94, SEP 2020. Web of Science Citations: 0.
VALEM, LUCAS PASCOTTI; GUIMARAES PEDRONETTE, DANIEL CARLOS. Graph -based selective rank fusion for unsupervised image retrieval. PATTERN RECOGNITION LETTERS, v. 135, p. 82-89, JUL 2020. Web of Science Citations: 0.
VALEM, LUCAS PASCOTTI; GUIMARAES PEDRONETTE, DANIEL CARLOS. Unsupervised selective rank fusion for image retrieval tasks. Neurocomputing, v. 377, p. 182-199, FEB 15 2020. Web of Science Citations: 0.
SANTANA, MARCOS C. S.; PASSOS, JR., LEANDRO APARECIDO; MOREIRA, THIERRY P.; COLOMBO, DANILO; DE ALBUQUERQUE, VICTOR HUGO C.; PAPA, JOAO PAULO. A Novel Siamese-Based Approach for Scene Change Detection With Applications to Obstructed Routes in Hazardous Environments. IEEE INTELLIGENT SYSTEMS, v. 35, n. 1, p. 44-53, JAN-FEB 2020. Web of Science Citations: 0.
GUIMARAES PEDRONETTE, DANIEL CARLOS; VALEM, LUCAS PASCOTTI; ALMEIDA, JURANDY; TONES, RICARDO DA S. Multimedia Retrieval Through Unsupervised Hypergraph-Based Manifold Ranking. IEEE Transactions on Image Processing, v. 28, n. 12, p. 5824-5838, DEC 2019. Web of Science Citations: 0.
CAMPOS, VICTOR DE ABREU; GUIMARAES PEDRONETTE, DANIEL CARLOS. A framework for speaker retrieval and identification through unsupervised learning. COMPUTER SPEECH AND LANGUAGE, v. 58, p. 153-174, NOV 2019. Web of Science Citations: 0.
SCARPARO, DANIELE CRISTINA; PINHEIRO SALVADEO, DENIS HENRIQUE; GUIMARAES PEDRONETTE, DANIEL CARLOS; BARUFALDI, BRUNO; ARNOLD MAIDMENT, ANDREW DOUGLAS. Evaluation of denoising digital breast tomosynthesis data in both projection and image domains and a study of noise model on digital breast tomosynthesis image domain. JOURNAL OF MEDICAL IMAGING, v. 6, n. 3 JUL 2019. Web of Science Citations: 0.
GUIMARAES PEDRONETTE, DANIEL CARLOS; WENG, YING; BALDASSIN, ALEXANDRO; HOU, CHAOHUAN. Semi-supervised and active learning through Manifold Reciprocal kNN Graph for image retrieval. Neurocomputing, v. 340, p. 19-31, MAY 7 2019. Web of Science Citations: 0.
VALEM, LUCAS PASCOTTI; DE OLIVEIRA, CARLOS RENAN; GUIMARAES PEDRONETTE, DANIEL CARLOS; ALMEIDA, JURANDY. Unsupervised Similarity Learning through Rank Correlation and kNN Sets. ACM Transactions on Multimedia Computing Communications and Applications, v. 14, n. 4 NOV 2018. Web of Science Citations: 1.

Please report errors in scientific publications list by writing to: