Bridging the gap between natural and medical images through deep colorization

Nowadays, Deep Neural Networks are widely used in medical image analysis, and they are the state-of-the-art in many applications. However, performances are often limited from the scarcity of labeled data since generating appropriated annotations requires medical experts and is very time-consuming. A large number of techniques are used to tackle this problem, and one of the most popular is to transfer learning from models pre-trained on ImageNet. Due to the different characteristics of the natural and medical domain, this strategy may be suboptimal. This thesis proposes three different colorization modules that learn how to map gray-scale medical images to three-channels colorful ones exploiting the classification loss generated during the training of a pre-trained model. Several experiments have been conducted to compare different transfer learning strategies from the RGB to the medical domain, with and without the proposed colorization module. Experiments were conducted on different X-ray datasets, including CheXpert, ChestX-ray14, and MURA. Results on the CheXpert dataset show that the colorization modules can improve the results up to 8% when the pre-trained model is frozen except for the last layer. This outcome suggests that the colorization module effectively compensates for the differences between the ImageNet and medical images. In general, fine-tuning the entire network obtains the best results, with or without the colorization module. The proposed strategy is most effective when the number of available samples is small (less than 512), whereas fine-tuning the entire network is the most effective strategy for large scale datasets.