Sampling of multi-modal distributions assisted with mixtures of normalizing flows

One of the core challenges in scientific computing is to have access to probability distributions which are too complex to be manipulated. The traditional answer to this computational issue lies in the Monte Carlo methods, which provide a way of approximating the moments of target probability distributions by sampling them, while avoiding the expensive numerical integration originally required. Despite the extremely significant contribution these methodologies have brought in computational science, there are still problems for which their performance is not efficient enough. This is the case of high-dimensional, multi-modal probability distributions. An interesting way to deal with this challenging task is to assist Monte Carlo sampling with machine learning techniques, such as the generative models, which are proved to be efficient in sampling complex high-dimensional probability distributions. This is the idea developed in the "Adaptive MC algorithm augmented with normalizing flows" proposed by Gabrié et al., 2022, where the traditional Monte Carlo local kernel is combined with a non-local one, parametrized by means of a generative model, the normalizing flow, which is trained on the fly with the generated samples. The aim of the work described in this Master's Thesis is to extend this adaptive algorithm in order to improve its performances when the modes of the target distribution have very different fine structures or statistical weights. The idea for dealing more efficiently with these conditions is to extend the original algorithm by replacing the unique normalizing flow with a mixture of multiple flows, where each component is trained to represent a single mode. From the analysis of the performances of this new algorithm and the comparison with existing methods, we prove that this strategy is accurate and efficient also in high-dimensional frameworks.