In deep learning research, enabling models to continuously and robustly acquire new knowledge represents a crucial step toward achieving more adaptive and efficient systems. Yet, when exposed to sequential streams of data, neural networks often struggle to maintain performance, suffering either from catastrophic forgetting (the erosion of previously acquired knowledge) or from loss of plasticity (LoP). This thesis specifically addresses the loss of plasticity phenomenon, marked by the progressive decline of a neural network’s representational capacity and the resulting inability to learn effectively from new data, particularly when training relies on traditional optimization methods, such as stochastic gradient descent (SGD) combined with backpropagation.