On the Challenges of Federated Learning in Semantic Segmentation across Domains

Artificial Intelligence is the technology behind self-driving cars, and it has already walked in our lives in ways that many of us are unaware of. The necessity to protect clients' privacy becomes a concern as a result of this stealthy assault. Federated Learning is a machine learning setting with the goal of preserving data privacy and ensuring data security. This scenario involves training statistical models on remote devices such as self-driving cars while keeping data localized. A central server aggregates the parameters from each local training client using a predefined algorithm, without having direct access to their data, which can be sensitive such as car routes. Following this, the updated global model is sent back to the client for iterative training. Semantic Segmentation, on the other hand, is a Computer Vision task that answers the questions "What is in this image, and where is it located in the image?". The goal of semantic segmentation is to assign a class label to each pixel of an image based on what is being represented. Convolutional Neural Network architectures in an encoder-decoder fashion are commonly used to accomplish this task. Segmentation models are useful for a variety of applications, including autonomous vehicles that must perceive their surroundings in order to interact safely with our existing roads. Both of these scenarios present a number of challenges, such as statistical heterogeneity in federated learning and the requirement for real-time full-resolution prediction for semantic segmentation. As a result, it is self-evident that combining these two settings may necessitate overcoming a number of challenges. The primary goal of this thesis is to develop a novel standard of Federated Learning in Semantic Segmentation for use in autonomous driving. In this framework, each car can be viewed as a client, with access to a personal dataset generated by on-board devices. Users can drive and collect data in a variety of locations, weather, and lighting conditions, so each domain distribution may differ from client to client, posing a number of challenges.