Decentralized Federated Learning for Object Detection Tasks - A Routing Approach to the Reduction of the Communication Overhead

Decentralized Federated Learning (DFL) is a new paradigm that arose to address the problems of having a central server in Federated Learning (FL). FL allows to train ML models in a privacy preserving way by sending model updates instead of the raw data to a central server for aggregation, but it presents issues with single point of failure and communication bottleneck at the server. DFL instead eliminates the need for a central server by exchanging the model updates directly between the participating clients. In order to reduce resource consumption, a small number of connections among the clients is preferable. However, the topology of the system plays an important role in the performance of the training task, especially under heterogeneous data distribution, and the elimination of connections can deteriorate it severely. This work presents an analysis of the impact of the topology of a DFL system on the training task, and proposes a routing algorithm for the selection of the minimum amount of connections necessary to reach all nodes while minimizing the impact on performance, considering extremely unbalanced non-Independently and Identically Distributed (i.i.d.) data among the clients. The analysis is based on the distribution of data generated by the partitioning of an Object Detection (OD) dataset, the MS COCO, and through simulations the proposed algorithm was shown to be effective in reducing the number of connections, up to eight times, while keeping a similar performance. A simple validation of the robustness of the system under the specific data distribution used is also provided, by simulating node failures at different points of the training task.