Alessandro Favero
Spectral Analysis of Infinitely Wide Convolutional Neural Networks.
Rel. Alfredo Braunstein, Matthieu Wyart. Politecnico di Torino, Corso di laurea magistrale in Physics Of Complex Systems (Fisica Dei Sistemi Complessi), 2020
|
Preview |
PDF (Tesi_di_laurea)
- Tesi
Licenza: Creative Commons Attribution Non-commercial No Derivatives. Download (1MB) | Preview |
Abstract
Recent works have shown the equivalence between training infinitely wide fully connected neural networks (FCNs) by gradient descent and kernel regression with the Neural Tangent Kernel (NTK). This kernel can also be extended to convolutional neural networks (CNNs), modern architectures that achieve stellar performance in image recognition, and other translational-invariant pattern detection tasks. The resulting Convolutional NTKs have been shown to perform strongly in classification experiments. Still, we lack a quantitative understanding of the generalization capabilities of these models. In this thesis, we introduce a minimal convolutional architecture, and we compute the associated NTK. Following recent works on the statistical mechanics of generalization in kernel methods, we study this kernel's performance in a teacher-student setting, comparing it with the NTK of a two-layer FCN when learning translational-invariant data.
Finally, we test our predictions with numerical experiments both on synthetic and real data
Relatori
Anno Accademico
Tipo di pubblicazione
Numero di pagine
Corso di laurea
Classe di laurea
Ente in cotutela
Aziende collaboratrici
URI
![]() |
Modifica (riservato agli operatori) |
