In recent times, we have more than ever the need to analyze systems with a large number of degrees of freedom. These may show a rugged energy landscape and even basic operations, like the minimization of a function, may become computationally expensive due to the aforementioned ruggedness. To tackle this problem, one of the most used algorithm is the gradient descent (GD). It is a fundamental tool in computer science but an increase in the complexity of the problems is shifting our interest toward optimized and more efficient versions of the 'vanilla' GD. A fundamental model in statistical mechanics that fits perfectly our framework is the pure spherical p-spin, that arises in the theory that stands behind glassy systems.