Interpretability techniques for a time series classification model used to predict Acute Kidney Injury episodes

Machine learning models tend to be affected by the black box issue, meaning that the more complex they are, the harder it is to explain their outputs. This thesis focuses on the interpretability of a neural network that analyses data coming from a digital biomarker to predict a medical episode known as the Acute Kidney Injury (AKI). AKI is a serious pathology that can affect patients during hospitalization in intensive care unit. Because it happens suddenly, it can be difficult to predict or prevent it. A team of researchers of the Politecnico di Torino successfully developed a model that can be used to predict such episodes. However, due to its complexity, it is not easy for clinicians to interpret and understand its outputs properly. In order to facilitate the understanding of our time series classification model, several interpretability techniques have been applied. We discuss the pros and cons of each technique providing a recommendation on how to augment the information presented to the end-user in order to improve their trust in the model classification. We also show how the same techniques can be used in a model assessment phase to better understand the limitations and the margin of improvements of the current model.