Dynamics of Gradient Descent in High-dimensional Non-convex Canyon Landscapes

Continuous constraint satisfaction problems (CCSPs) describe systems with continuous degrees of freedom subject to constraints, and they appear in diverse domains such as artificial neural networks and confluent biological tissues. In this work, we investigate a mean-field model that captures a rigidity transition in confluent tissues, governed by random nonlinear equality constraints. We extend this model by introducing multiple replicas of the system and studying their evolution under gradient descent dynamics. Using dynamical mean-field theory (DMFT), we analyze how correlations between replicas evolve in the overparametrized regime, where solutions are still satisfiable. Our results show that inter-replica correlations relax toward nonzero asymptotic values, indicating that gradient descent dynamics retain memory of initial configurations. The scaling behavior of these correlations reveals Lyapunov-like sensitivity to initial overlaps, providing a window into the geometry of the underlying energy landscape. This study contributes to a deeper understanding of optimization dynamics in high-dimensional constrained systems and offers a controlled framework for exploring the landscape structure.