Uncertainty quantification in biomedical Deep Learning models

Sleep disorders are a serious public health problem with both psychological and physical effects. To formulate the diagnosis of such disorders, it is required that the patient sleeps in a clinic, while various biological signals are recorded: this set of signals is called polysomnography (PSG). PSG analysis is a work done by an experienced clinician by dividing the recording into epochs of 30 seconds and by giving a score to each of these epochs. The score corresponds to five different sleep stages: W that represents the waking stage to the drowsiness, N1 the beginning of the sleep, N2 before the deep sleep, N3 deep sleep, or R the dreaming phase. This procedure is called sleep scoring and is regulated by specific manuals. Sleep scoring is a slow, tedious and costly procedure in terms of clinician's time and therefore it is a cost to the hospital, which is why over the years more and more artificial intelligence-based solutions for automatic scoring have been proposed. Although the performance of machine and deep learning algorithms for automatic scoring have reached levels of accuracy comparable with those of a human scorer, they are not being introduced into daily practice. One of the reasons why clinicians do not use artificial intelligence systems for sleep scoring is related to trust in the systems themselves. In fact, it has been shown how systems such as neural networks tend to be overconfident and uncalibrated. The purpose of this thesis is to study methods for uncertainty quantification applied to deep learning for sleep scoring. Data from the open source Sleep-EDF Sleep Cassette database, particularly from healthy subjects, are used in the experiments, while a convolutional network called DeepSleepNet-Lite appropriately trained is used as a classifier. The gold standard for quantifying uncertainty in neural networks is called Maximum Class Probability (MCP) and is obtained by selecting the maximum value from the output of the last softmax layer. Experiments show the comparison between MCP and True Class Probability (TCP), a new method that exploits a specially trained auxiliary network, estimating the confidence of predictions. The construction of the auxiliary network can be done in several ways, changing the training procedure, the loss function and the number of layers from which it is composed. In this thesis, it is shown how the auxiliary network can also be applied to neural networks for sleep evaluation. Moreover, this approach is a more flexible and efficient solution in terms of model calibration than the gold standard.