Deep Learning techniques for Natural Language Processing: A multilingual encoder model for NLI task

In this work, we build an Artificial Intelligence model that understands inference relations between English and Italian sentences. To do so, we leverage pre-trained Transformer-based sentence Encoders to understand and encode sentences into high-dimensional vectors. Then, we build and test different algorithms for comparing the encoded vectors and inferring relations between texts. We start from the simple and fast dot-product (that obtains an accuracy of 50.13% on the SNLI validation set), we fine-tune the encoder (obtaining a 15.26% increment) and we move to more complex algorithms: Support Vector Machines (that obtain an accuracy of 84.13% on the same set). At the end, we study 4 different Deep Learnig end-to-end models with 4 different attention heads (Fully Connected, Convolution, Convolution generalization and dot-product generalization) We detail the theory behind Transformers and why they outperformed state-of-the-art architectures on Natural Language Processing tasks. We will stress the symmetries we exploited and the motivations that led us to improve models’ performances on two Italian datasets (obtaining an accuracy of 63.38% and 81.65% on Italian RTE-3 and ABE_ABSITA). While the model developed may still improve, it is already able to support quantitative text analysis in industrial environments.