In the scenario of deep learning models applied to Natural Language Processing (NLP), the Transformer architecture has brought great interest thanks to its attention mechanism, which helps models to focus on specific parts considering the relationship between words, independently of where they are placed in a sentence. Many models based on this type of architecture have been developed by the research communities. In my Master Thesis I examined the following language models: • Bidirectional Encoder Representations from Transformers (BERT) • Efficiently Learning an Encoder that Classifies Token Replacement Accurately (ELECTRA) • Generative Pre-Trained 2 (GPT-2) These models have been applied to the NLP text classification task on different datasets both in English and in Italian in order to evaluate and compare the performances obtained with the models pre-trained on an English corpus rather than with a multilingual model.