Evaluating the entropy of physical systems using diffusion models

This work focuses on the application of diffusion models, a very well-known type of generative artificial intelligence, to the mathematical and physical problem of the computation of entropy. The main goal of this work is to show that diffusion models can be used to compute entropy in a very efficient way, exploiting their recognized ability to learn the underlying data distrbution of a system. To this scope, first of all a good part of the presented work has been dedicated to the study of similar methods present in literature, to understand in particular which datasets are used to benchmark the task of entropy computation. This analysis brought to the result that in the physics world the preferred choice is to focus on spin systems, such as the Ising and XY models, for which the entropy has been analytically computed. Hence, these systems have been studied and methods to create samples from them have been implemented to create the corresponding datasets on which to train the diffusion models. Moreover, an in depth study of the history and development of diffusion models to understand their functioning and how to exploit their capabilities has been carried out, with particular attention to the data preparation and to what is called the score network. To implement this last function a transformer architecture has been chosen for its versatility and powerful computational properties, that respondend well to the necessities given by the structure of the data. Diffusion models are a fairly recent technology, and to exploit their full potential it is important that all people in the scientific community, even if not specialized in machine learning or data science, see the great benefits that the application of this technology may bring. It was for this reason and prove the applicability of this type of generative AI in a very wide variety of fields, that the choice was made to apply diffusion models to the task of entropy computation, where they had never been employed before. Entropy computation is a well-known problem in science since this quantity can give a lot of information on the data one is treating, but unfortunately it is often unfeasible to analytically compute it for systems or datasets. Up to now, few attempts have been made to use deep learning techniques to estimate this quantity. The objective of this thesis has been to implement and test the proposed method employing diffusion models: all the code developed for the present work, including the entropy computations, the scripts for dataset generation and visualization, are available at this GitHub repository.