OPTIMIZATION OF CONVOLUTIONAL NEURAL NETWORKS TRAINING FOR FEDERATED LEARNING ON EMBEDDED SYSTEMS

The advancement of low power technologies and the improvement of wireless communication systems and infrastructures, fueled the Internet of Things (IoT), enabling the proliferation of connected sensors able to collect and transmit data over the internet. Meanwhile, thanks to the recent breakthroughs in Artificial Intelligence (AI), Convolutional Neural Networks in particular, computers can learn trends from the collected data and extract meaningful insights to make decisions autonomously. IoT and AI are twin pillars of a new revolution: the Artificial Intelligence of Things (AIoT). This revolution poses several opportunities and challenges. The availability of large-scale datasets generated by pervasive networks of sensors enabled the development of AI models achieving unprecedented accuracy in many domains, e.g., computer vision and natural language understanding. At the same time, the creation and management of centralized datasets are raising several privacy and security concerns. For example, security cameras, smart speakers, and smartphones collect sensitive information that users might be unwilling to share with service providers. Therefore, the major challenge today is to develop a new class of learning strategies that enable to process data locally, i.e., on embedded systems at the edge of the IoT. This is the goal of Federated Learning (FL), the target of this work, an emerging learning paradigm where the model training is distributed over a connected fleet of devices. Each device uses its own data to train a local version of the model; periodically, the devices send their model versions (instead of the data) to a centralized server; here, the local versions are aggregated in a global model, that is sent back to the edge devices. Although previous research demonstrated that training from decentralized data is feasible, at least for what concerns the model accuracy, how to manage the limited energy resources of the IoT end-nodes remains an open problem. Indeed, FL involves frequent upload and download of models to achieve competitive accuracy, introducing a massive communication overhead, which shortens the battery life of the edge devices. This work introduces an optimized training strategy that gradually reduces the number of model parameters that are trained and synchronized with the global model. Experimental results collected on standard datasets demonstrated that the benefits of the proposed strategy are twofold: (i) substantially reduce the communication overhead at the same accuracy, with savings ranging from 14% to 59% (depending on the accuracy levels and on the dataset) with respect to a standard FL; (ii) increase the number of model updates at the same communication cost, thus improving the model accuracy, up to +2.5%.