Machine Learning is increasingly being leveraged in business processes to make automated decisions. Nevertheless, a decision is rarely made by a standalone machine learning model, but is rather the result of an orchestration of predictive models, each predicting key quantities for the problem at hand, which are then combined through decision rules to produce the final decision. For example, a mobile phone company aiming to reduce customer churn would use machine learning to predict churn risk and rank potential retention offers, and then apply eligibility rules and other policies to decide whether a retention offer is worth proposing to a certain customer and, if so, which one.