Adversarial learning in natural language processing

Abstract

Machine learning techniques have been paramount in the latest years, mainly due to their high effectiveness in computer vision and Natural Language Processing (NLP) problems. Despite their significant discriminatory power, their generative capacities are far from having an ideal performance in specific applications, often related to text-data mining ones. The most significant setback occurs when those techniques are employed with discrete representations such as characters and words sequences. Although some novel works presented feasible solutions to the natural language generation problem, most could not generate a completely `natural´ language, lacking from either morphological, syntactical or semantical knowledge. Furthermore, another recurring issue in the NLP area concerns the scarcity of data to feed deep learning architectures. Popularly used in the image processing field for generating synthetical data, adversarial learning has not yet been developed to a satisfactory point in the text-data area, producing lots of repeated words and even some without semantical significance. Therefore, this present proposal endeavors on fostering forefront research in developing adversarial models in the context of natural language processing. Generative Adversarial Networks (GANs) and Recurrent Neural Networks (RNNs), with particular attention to Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTMs) architectures, will be studied. This proposal also comprises an internship at Stanford University, United States. (AU)