polito.it
Politecnico di Torino (logo)

Using random subspace training to investigate deep neural networks

Lorenzo Pacchiardi

Using random subspace training to investigate deep neural networks.

Rel. Alfredo Braunstein. Politecnico di Torino, Corso di laurea magistrale in Physics Of Complex Systems (Fisica Dei Sistemi Complessi), 2018

Abstract:

Despite their ubiquitous applicability and stunning performances, neural networks lack of understanding and interpretability. Many research efforts are nowadays dedicated to tackle this tasks from a theoretical point of view, trying to explain empirical observations on the improvements given by certain architectures and techniques, and hoping that a better comprehension may lead to faster and more performing algorithms.This work is inserted in this research track. An approach to investigate the importance of the abundant number of parameters of a network, is presented, based on the estimation of the intrinsic dimension of the objective landscape by restricting the training algorithm to a random lower-dimensional subspace of the full parameters space. Subsequently, the same technique is applied separately to the parameters of every network layers. In this way, the contribution to the final performance of the network of each layer can be analyzed, and the number of degrees of freedom necessary for it to perform its function can be found. Moreover, such investigations can provide insights on the properties of the objective landscape; specifically, it was experimentally observed that these properties are quite similar across the hyperplanes spanned by the layers' parameters, independently on the type of transformation performed by the layer or its position in the architecture. This suggests that the number of degrees of freedom in the network may be more important than their specific position.

Relatori: Alfredo Braunstein
Anno accademico: 2017/18
Tipo di pubblicazione: Elettronica
Numero di pagine: 49
Informazioni aggiuntive: Tesi secretata. Full text non presente
Soggetti:
Corso di laurea: Corso di laurea magistrale in Physics Of Complex Systems (Fisica Dei Sistemi Complessi)
Classe di laurea: Nuovo ordinamento > Laurea magistrale > LM-44 - MODELLISTICA MATEMATICO-FISICA PER L'INGEGNERIA
Ente in cotutela: Université de Paris-Sud (Paris XI) (FRANCIA)
Aziende collaboratrici: NON SPECIFICATO
URI: http://webthesis.biblio.polito.it/id/eprint/8029
Modifica (riservato agli operatori) Modifica (riservato agli operatori)