A Multimodal Visual Language Model for Musical Encoding

This thesis describes the main steps of a research experience conducted at LINKS foundation (Torino, Italy), regarding the implementation of a multimodal visual language model for musical encoding. The goal of this activity was testing the powerful tools of AI generative modelling, in order to better understand the potentials of machine learning in the art field. From this perspective, the thesis shows how, in particular, a denoising diffusion probabilistic model (DDPM) can be adopted to generate artistic images from a 30 second musical input. The data preparation (collection, categorization, normalization, etc.) as well as the research and the implementation of a suitable model have been the fundamental passages around which all the project was developed.