In the last decade, Natural Language Processing (NLP) has made giant leaps in treating textual data, by means of large deep neural networks capable of addressing complex tasks such as machine translation, language generation and text summa- rization. These models are based on the principle of distributionality — a word’s meaning is defined by the context in which it appears — and learn a represen- tation of words in a vector space. This implies training the model on a massive amount of data to learn the interdependencies among words. In addition, despite the successful achievements, the general problem of natural language understanding is still unsolved as neural network models lack of explainability, which raises further concerns when we entrust our decisions to their predictions in critical domains.