VAE for NLP: novel architectures and possible applications

Natural Language Processing consist in the study of how to automatically process information conveyed in Natural Language, that is, human language.This thesis is interested in a subset of this domain called Text Generation which consist in the elaboration of computational system capable of generating text comprehensible by humans.More specifically, it is interest in explore a specific model named Variational Autoencoders, which is one of the most popular models in the academia in the time of elaboration of this work. In the context of the VAE, this thesis aims to accomplish two sets of goals.The first is: to improve the overall performance of the VAE in the text generation task, the understanding of the most new techniques and reflect upon how to evaluate Text Generation Models. This shall be done by combining recent advances in the VAE architectures and by proposing a new technique. The recent advances consist in performing Aggressive Training, using Length as a feature and adding a Bag of Word loss. The new technique consist in splitting the representation of the VAE in two disentangled vectors: one for content and other for grammar. The second set of goals is to explore the applications of the text generation system to other tasks, analyzing its performance and giving guidance to future work in the application of VAEs and Text Generation Systems. The applications in question are Information Retrieval and Text Autocomplete. The goal is not to make tailored system, but explored how a VAE could be used in such context.