Politecnico di Torino (logo)

Optimization of the gradient descent dynamics in simple mean field spin glasses.

Yuri Lombardo

Optimization of the gradient descent dynamics in simple mean field spin glasses.

Rel. Andrea Pagnani, Pierfrancesco Urbani. Politecnico di Torino, Corso di laurea magistrale in Physics Of Complex Systems (Fisica Dei Sistemi Complessi), 2021

PDF (Tesi_di_laurea) - Tesi
Licenza: Creative Commons Attribution Non-commercial No Derivatives.

Download (7MB) | Preview

In recent times, we have more than ever the need to analyze systems with a large number of degrees of freedom. These may show a rugged energy landscape and even basic operations, like the minimization of a function, may become computationally expensive due to the aforementioned ruggedness. To tackle this problem, one of the most used algorithm is the gradient descent (GD). It is a fundamental tool in computer science but an increase in the complexity of the problems is shifting our interest toward optimized and more efficient versions of the 'vanilla' GD. A fundamental model in statistical mechanics that fits perfectly our framework is the pure spherical p-spin, that arises in the theory that stands behind glassy systems. It presents a rugged energy landscape but at the same time it is somewhat easy to analyze, allowing us to obtain some insights by studying it. In this model, the system is characterized by spherical spins affected by long range p-body interactions, which are weighted by random interaction coefficients. What emerges in the pure p-spin is that the number of stationary points of the Hamiltonian increases exponentially with the energy up to a certain threshold. Thus as we try to make our way toward lower energies using a GD algorithm the number of local minima becomes so numerous that, in the macroscopic limit, our descent gets stuck at this threshold value. These results of the pure p-spin were considered to be universal, but a recent study showed us that it is not the case, for example in the case of a mixed p-spin model. The latter is structurally identical to the pure one, the only difference is that we have to consider interactions that can affect a different number of spins simultaneously. In these new settings, we are able reach energies lower than the threshold by modifying wisely the initial conditions of the spins and this phenomenon leads to a dynamical phase transition. Following these results, we wonder if there are other ways to cross the threshold level, and thus if we could optimize even more the descent on a particular mixed p-spin model. We enrich our problem by adding a new function v(t) to the Hamiltonian which leads to an optimal control problem. The purpose of this new element is to reach a lower energy value, at the end of our simulation, compared to the standard algorithm. To build this optimized function, we write the Lagrangian of the GD evolution, by adding suitable auxiliary variables, which leads to a set of integro-differential equations that allows us to estimate the control function v(t). We perform a numerical integration of these sets of equation and then investigate under which conditions the algorithm performs better than the standard gradient descent.

Relators: Andrea Pagnani, Pierfrancesco Urbani
Academic year: 2021/22
Publication type: Electronic
Number of Pages: 38
Corso di laurea: Corso di laurea magistrale in Physics Of Complex Systems (Fisica Dei Sistemi Complessi)
Classe di laurea: New organization > Master science > LM-44 - MATHEMATICAL MODELLING FOR ENGINEERING
Aziende collaboratrici: CEA CESTA
URI: http://webthesis.biblio.polito.it/id/eprint/20439
Modify record (reserved for operators) Modify record (reserved for operators)