Machine learning models tend to be affected by the black box issue, meaning that the more complex they are, the harder it is to explain their outputs. This thesis focuses on the interpretability of a neural network that analyses data coming from a digital biomarker to predict a medical episode known as the Acute Kidney Injury (AKI). AKI is a serious pathology that can affect patients during hospitalization in intensive care unit. Because it happens suddenly, it can be difficult to predict or prevent it. A team of researchers of the Politecnico di Torino successfully developed a model that can be used to predict such episodes.