Responsible Artificial Intelligence for Critical Decision-Making Support: A Healthcare Scenario

Artificial intelligence is being implemented in an increasing number of areas, among which healthcare, finance and justice, with the aim of using it as decision-making support. However, high-performing models are usually not trusted by the final user, since they are not transparent. Understanding the model behaviour is fundamental, especially in critical tasks, but it is not enough to build AI systems that have the well-being of humans in the first place. To understand what AI needs to be trustworthy, the concept of responsibility in the Artificial Intelligence field is introduced. It is analyzed in detail, considering all the components needed to build a Responsible Artificial Intelligence system, from explainability to fairness, passing through accountability, security, inclusiveness and reliability. The thesis addresses the problem of the lack of interpretability of models making use of the Responsible AI Toolbox (RAIToolbox), built by Microsoft. The thesis aims also to understand and analyze the strengths and weaknesses of this toolbox, performing different experiments. RAIToolbox leverages explanation methods to help humans during the decision-making process, inspecting the reasons behind model predictions. It makes use of post-hoc techniques to analyze the behaviour of models from both the perspective of data subgroups and individual instance predictions. The analyzes are built on the notion of cohorts, which are the combination of feature-value pairs intrinsically interpretable. They capture relevant associations, defining subgroups in the features domain. Making use of cohorts allows the tool to be used in spite of the algorithm, and virtually it can be applied to any model for supervised prediction. Among the multiple components of the RAIToolbox, the Error Analysis one addresses the problem of identifying and analyzing data subgroups in which a model behaves differently. This allows to evaluate model fairness, identify biases and test the model via a comparison between the behaviour of the model on different cohorts and the behaviour on the overall dataset. From the perspective of individual instances, Counterfactual Analysis tool explains the prediction of any model on a specific instance by analyzing what perturbations of single features or joint feature subsets are needed to cause the model to change the prediction. Individual instances analysis, together with critical cohorts analysis, allows to better understand the results of the model Fairness Analysis. The toolbox is tested on the COMPAS dataset, showing its effectiveness in revealing the model's behaviour at both cohort and individual instance levels. After this validation phase, the RAIToolbox is used to analyze two different scenarios' medical datasets. The former is about 101766 diabetics patients, characterized by 48 features and its target is to predict patients' readmission time to the hospital. The latter is about myocardial infarction complications and contains information about 1700 patients characterized by 123 features and the aim is to predict causes of death if any. Both datasets contain sensitive features and are used to train classification and regression models, allowing the analysis of the different tasks' models. All the performed analyzes allow an understanding of how the RAIToolbox can be used to help physicians better trust models and take responsible decisions. Moreover, the tools allow identifying the strengths and weaknesses of the datasets in the scenario where they were considered.