On the influence of Markovian models for restricted Boltzmann machines training

Abstract

Deep learning techniques have been widely investigated by the machine learning community in the last years, mainly due to their suitable results in crucial applications, such as face recognition and speech identification, as well as object classification. One of the most used techniques is the Restricted Boltzmann Machine (RBM), which can be defined, essentially, as being stochastic neural networks that aim at estimating the weights between different layers using Markov chain (MC) sampling techniques. Currently, a considerable number of works have focused on studying such sampling methods, since the efficiency and effectiveness of an RBM is strongly related to them. Therefore, this proposal aims at investigating three approaches for MC sampling: Contrastive Divergence, Persistent Contrastive Divergence and Fast Persistent Contrastive Divergence. Additionally, we aim at comparing them in different scenarios concerning image classification tasks.