Combining Split and Federated Architectures for Efficiency and Privacy in Deep Learning

Mobile phones and wearable devices are used and carried everywhere by people, producing every day a huge amount of distributed and sensitive data. Standard machine learning techniques need to collect this kind of data all on the same machine to compute the training and obtain a model useful to make predictions. To better preserve the privacy of data and, at the same time, guarantee a comparable performance, Federated and Split Learning have been designed. This work tries to improve the efficiency and privacy of Split and Federated learning combining them in two different types of architectures using the PySyft library implemented inside PyTorch. The code has been developed to easily switch from local to remote learning, simulating the distributed process inside the same machine and, then, running the neural network on workers located on different devices. Then, a trade-off analysis between the two architectures in terms of efficiency and privacy was performed: changing the distribution of data inside each device, estimating the possibility to rebuild the original data using the neural network inversion attack, and exploring possible privacy improvements. Finally, the results encourage us to further investigate the architectures and find which ones are more suitable to maximize both the performance and privacy, according to the data.