Optimization of the gradient descent dynamics in simple mean field spin glasses.

In recent times, we have more than ever the need to analyze systems with a large number of degrees of freedom. These may show a rugged energy landscape and even basic operations, like the minimization of a function, may become computationally expensive due to the aforementioned ruggedness. To tackle this problem, one of the most used algorithm is the gradient descent (GD). It is a fundamental tool in computer science but an increase in the complexity of the problems is shifting our interest toward optimized and more efficient versions of the 'vanilla' GD. A fundamental model in statistical mechanics that fits perfectly our framework is the pure spherical p-spin, that arises in the theory that stands behind glassy systems. It presents a rugged energy landscape but at the same time it is somewhat easy to analyze, allowing us to obtain some insights by studying it. In this model, the system is characterized by spherical spins affected by long range p-body interactions, which are weighted by random interaction coefficients. What emerges in the pure p-spin is that the number of stationary points of the Hamiltonian increases exponentially with the energy up to a certain threshold. Thus as we try to make our way toward lower energies using a GD algorithm the number of local minima becomes so numerous that, in the macroscopic limit, our descent gets stuck at this threshold value. These results of the pure p-spin were considered to be universal, but a recent study showed us that it is not the case, for example in the case of a mixed p-spin model. The latter is structurally identical to the pure one, the only difference is that we have to consider interactions that can affect a different number of spins simultaneously. In these new settings, we are able reach energies lower than the threshold by modifying wisely the initial conditions of the spins and this phenomenon leads to a dynamical phase transition. Following these results, we wonder if there are other ways to cross the threshold level, and thus if we could optimize even more the descent on a particular mixed p-spin model. We enrich our problem by adding a new function v(t) to the Hamiltonian which leads to an optimal control problem. The purpose of this new element is to reach a lower energy value, at the end of our simulation, compared to the standard algorithm. To build this optimized function, we write the Lagrangian of the GD evolution, by adding suitable auxiliary variables, which leads to a set of integro-differential equations that allows us to estimate the control function v(t). We perform a numerical integration of these sets of equation and then investigate under which conditions the algorithm performs better than the standard gradient descent.