Anatomically constrained Cross-domain CT image translation using CycleGAN

This study describes a method to perform an image to image translation by means of a deep generative model. Its purpose is to obtain, from abdomen CT images with no contrast, contrast enhanced CTs and vice versa. The entire study was carried out through a six-month internship in Paris at the Ecole Nationale Supérieure des Télécommunications (TELECOM Paris), in the department of Image, Data, Signal (IDS). The medical images are obtained from different databases that are available online, and they specifically analyse only the abdominal component of patients. Starting from the lungs to the femoral head. In particular, the generative model is composed of Cycle Consistent Adversarial Networks (called CycleGAN), whose main properties are based on the recent model described by Zhu et al. Where they describe two generators, based on deep neural networks, able to generate new images starting from two unpaired sets of images with different characteristics. One of the main problems related to medical imaging is the small number of images in the databases, furnished by hospitals or available online in open-access, which is usually not enough to accurately train deep learning methods. The lack of large data-sets in medical imaging, with respect to other branches of Computer Vision, is due to the fact that acquiring medical images is usually expensive and difficult and that in many cases images can not be distributed in open-access for privacy reasons. To help identify pathologies, doctors often inject a contrast agent, e.g. iodine or barium-based, which is absorbed and then evacuated by the body. Injected images better highlight the interface between different tissues and structures (e.g., kidney/tumor), facilitating the identification of possible pathologies in the patient and are therefore usually preferred to non-injected images for segmenting anatomical organs for surgery planning or research purposes. Unfortunately, however, for many patients the use of contrast agents is not possible due to possible complications: like allergic reactions or chronic pathologies. This obviously makes it more difficult to diagnose the disease and limits the production of certain types of images. For this reason, this study describes a method on which we try to artificially generate medical images with the contrast, even though it had never been injected into the patient, increasing the number of these types of images. At the same time we apply the inverse transformation, obtaining contrast-free CT images, which can be used to increase the number of available data sets with images that have an excellent ground truth. The translation is done using only healthy patients, in this way we prevent the generation of artefacts of any pathologies, such as thrombosis or stenosis, which are only visible in the images with contrast inserted, because in the image without contrast there is greater homogeneity between different anatomical structures. For this reason the generation of pathological elements may be difficult for a doctor to interpret. The use of databases with patients with pathologies may be the baseline for a future study.