3D Facial Animation Based in Structural Registration

Abstract

This project will be developed during the Master's degree of student Andrea Britto Mattos, under the guidance of Professor Roberto Marcondes Cesar Junior, being part of the areas of Computer Graphics, Pattern Recognition and Computer Vision. This project will extend a system developed during the candidate's Undergraduate education in the Department of Computer Science of IME-USP. This system can manipulate three-dimensional faces that can pronounce sentences in Portuguese and move its eyes according to the position of the user's face, when it is detected by a camera. The project aims to extend the previous work, which can only support limited models and facial expressions, in order to allow the use of generic models of real people obtained via 3D scanners. To achieve this, it will be necessary to perform the registration of these faces, creating a correspondence between the vertices of two generic meshes, so they can be interpolated and animated by the computer.A new technique, based on structural methods will be studied and adapted so it can help solve the problem of registration of these faces. This approach is inspired by a study of pattern matching in gel electrophoresis, currently applied in 2D images. The goal of this project is to extend this technique so it can be used for three-dimensional models, evaluating the quality of this new approach. The study of structural matching of 2D electrophoresis gels has been done in the Vision and Image Processing Research Group of IME-USP, which the candidate is a member of.In this group, there is also a study on the reconstruction of the geometric structure of faces captured by 3D scanners. Therefore, the project also aims the collaboration between the three projects.