Pietro Cagnasso
Efficiency and Generalization in Federated Learning: Insights from Sharpness-Aware Minimization.
Rel. Barbara Caputo, Debora Caldarola, Marco Ciccone. Politecnico di Torino, Corso di laurea magistrale in Data Science And Engineering, 2023
Abstract
Federated Learning (FL) is a distributed Machine Learning approach that enables the training of neural networks across multiple devices (i.e., clients) owning privacy-protected data. In FL, only model parameters are exchanged during training, ensuring that sensitive user data remains protected. However, in real-world scenarios, the clients’ data often exhibits non-i.i.d. characteristics, stemming from individual behaviors and geographical locations, which can, in turn, lead to slower and unstable optimization processes. Recent research has addressed this challenge by focusing on the loss landscape, drawing from the connection between converging toward flatter minima and achieving better generalization. Specifically, it was shown how leveraging Sharpness-Aware Minimization (SAM) during local training, which aims at finding minima having both low loss value and low sharpness, enables federated global models to reach optimal results.
However, these works present three main flaws: (i) SAM doubles the computations over traditional methods, which is costly for resource- and battery-constrained devices; (ii) the advanced approaches considering global sharpness also double the communication costs; (iii) these works overlook recent SAM variants that offer improved performance or cost reduction
Relatori
Anno Accademico
Tipo di pubblicazione
Numero di pagine
Informazioni aggiuntive
Corso di laurea
Classe di laurea
URI
![]() |
Modifica (riservato agli operatori) |
