Hassine El Ghazel
Decomposing the Modality Gap: Single-Axis Attribution of Representation Geometry in Late-Fusion Vision-Language Models.
Rel. Giuseppe Rizzo, Luca Catalano, Federico D'Asaro. Politecnico di Torino, Corso di laurea magistrale in Data Science And Engineering, 2026
|
Preview |
PDF (Tesi_di_laurea)
- Tesi
Licenza: Creative Commons Attribution Non-commercial No Derivatives. Download (10MB) | Preview |
|
|
Archive (ZIP) (Documenti_allegati)
- Altro
Licenza: Creative Commons Attribution Non-commercial No Derivatives. Download (905B) |
Abstract
Late-fusion vision-language models use a frozen visual encoder and a frozen large language model (LLM) linked by a trainable projector. In the embedding space, the image and text embeddings can take different locations in the space. This gap is called the modality gap and summarised by early work as the distance between the centroids of the two modality clouds. This thesis views the gap differently as a composite object of different axes, and we test on each axis to interpret its effect on image captioning and how well generated captions are grounded in their images. We separated the gap into four measurable axes: the centroid distance, the total variance of the image cloud, the number of directions over which that variance is spread, and the alignment of the two subspaces.
A diagnostic model is later trained to move one axis at a time while pinning the remaining three, and its effect on the quality of captions is then interpreted against a baseline with all axes pinned
Relatori
Anno Accademico
Tipo di pubblicazione
Numero di pagine
Corso di laurea
Classe di laurea
Aziende collaboratrici
URI
![]() |
Modifica (riservato agli operatori) |
