The use of predictive automated decision tools has seen a rapid increase in recent years, with the capability of producing accurate predictions in various domains. While new ideas and algorithms aim to improve the performance of these models, there is growing concern among the machine-learning community about the reliability of the predictions they produce. This has given rise to the new field of Trustworthy Artificial Intelligence. This thesis investigates the issue of Trustworthy AI in the context of predicting Acute Kidney Injury (AKI) concerning fairness and robustness aspects. To assess fairness, AIF360 metrics and a bias mitigation algorithm were applied, revealing that patients with chronic kidney disease or chronic heart failure are more likely to develop AKI.