Towards more stable continuous-time functional diffusion processes

Continuous-time functional diffusion processes demonstrated great potential in the generation of resolution-invariant data and in generalising diffusion models to different data types. However, training these models is challenging due to the high number of hyperparameters and the instability of the training process. Meta-learning is the dominant approach for training this kind of models. Two sets of parameters are used, where one specialises to the task at hand, while the other is computed at inference time to adapt the network for the current datum through few iterations of stochastic gradient descent. In this thesis, we propose a more stable approach to infer functional representations of data and avoid the pitfalls of meta-learning. We employ a different neural network to infer the set of parameters that specialises to the datum. This allows to preserve important properties, such as resolution invariance in case of visual tasks. Additionally, we derive a new functional Stochastic Differential Equation (SDE) that is more stable and requires fewer hyperparameters. Specifically, we remove the drift term by showing that the infinitesimal generator can be set to the zero operator without losing the guarantee for the existence of the reverse diffusion process. Moreover, we show that it is possible to estimate the covariance operator of the Brownian term of the diffusion process directly from data as the empirical covariance of the dataset. The proposed approach allows to reduce the number of hyperparameters, leading to a faster, simpler and cheaper training process. Remarkably, this new functional SDE resembles the variance exploding SDE, sharing similar properties, such as the unbounded variance in the forward diffusion process in the limit of infinite time. Together, the two main contributions of this work provide a more stable and efficient way to train continuous-time functional diffusion processes. We validate the proposed method on a set of experiments, showing that it is more stable and requires fewer computational resources compared to the state of the art of continuous-time functional diffusion processes. The implementation of this project was done using the PyTorch library, expanding and translating the codebase of the paper Continuous-Time Functional Diffusion Processes, originally written using Jax. Future work will be done to demonstrate the ability of the proposed method to scale to more complex datasets and bigger architectures.