Nowadays, Deep Neural Networks are widely used in medical image analysis, and they are the state-of-the-art in many applications. However, performances are often limited from the scarcity of labeled data since generating appropriated annotations requires medical experts and is very time-consuming. A large number of techniques are used to tackle this problem, and one of the most popular is to transfer learning from models pre-trained on ImageNet. Due to the different characteristics of the natural and medical domain, this strategy may be suboptimal. This thesis proposes three different colorization modules that learn how to map gray-scale medical images to three-channels colorful ones exploiting the classification loss generated during the training of a pre-trained model.