Federated Visual Geo-Localization

Visual Geo-Localization is the problem of estimating the geographical coordinates of an image given a database of geo-tagged pictures from known places. This task carries out as an image matching and retrieval problem. Where the picture to be localised compares itself against all the images in the database and a shortlist of possible matches retrieved with a k-Nearest Neighbor (kNN) search. The matching operation uses global image descriptors. The task poses some concerns related to the centralised maintenance of the images and privacy. (e.g. A violation of users' privacy is done if we collect their photographs. Due to photos loaded on a central server, they will be available for everyone.) These issues are at the foundation of the Federated Learning paradigm. In Federated Learning, instead of having all the data and the computational power on a single device, these are spread across multiple machines. Each machine corresponds to a client, and all the clients communicate to a central server. At each iteration, the clients are doing local training, and they send to the server the local updated model, the server will aggregate them and send back the updated global model. This thesis aims to investigate, for the first time, the formulation of the Visual Geo-Localization task in a Federated Learning setting, considering a possible application to the scenario of autonomous driving cars. When the cars do not know the current location, they will discover using an image retrieval technique. Considering that an automobile has small computational power and the availability of cars across the globe is higher, Federated Learning will fit perfectly. In this analysis, Federated Learning is compared to a centralised approach, trying to obtain the best result possible in a basic Federated Learning paradigm. Some of the main challenges posed by the Federated Learning setting, such as the Statistical Heterogeneity and the Data Imbalance problems for the task of Visual Geo-Localization, have been analysed in depth.