Despite their ubiquitous applicability and stunning performances, neural networks lack of understanding and interpretability. Many research efforts are nowadays dedicated to tackle this tasks from a theoretical point of view, trying to explain empirical observations on the improvements given by certain architectures and techniques, and hoping that a better comprehension may lead to faster and more performing algorithms.This work is inserted in this research track. An approach to investigate the importance of the abundant number of parameters of a network, is presented, based on the estimation of the intrinsic dimension of the objective landscape by restricting the training algorithm to a random lower-dimensional subspace of the full parameters space.