Semantic Segmentation of Point Clouds for Urban Mapping: presentation and benchmarking of Turin3D Dataset

In recent years, considerable interest has emerged in improving urban planning and management, which has led to the need to explore new technologies. In particular, there has been a focus on the use of aerial aircraft for data acquisition in the urban environment. Although it is possible to acquire a large amount of data through these techniques, these are usually collected by LiDAR sensors, a technology widely used in applications using 3D data. The real problem is how to process and interpret them to extract useful knowledge in application contexts, so the use of these data collected by LiDAR sensors has led to the opening of new research scenarios in this field. The aim of this thesis is therefore to address the segmentation challenges inherent to the processing of 3D data acquired by aircraft operating in urban contexts, specifically obtain good results by testing different models on a new dataset containing data collected on a limited area of the city of Turin (about 1.43 km^2, divided into 57 blocks of about 25,000 m^2 each). The data were collected by a LiDAR sensor during an aerial flight on 29 January 2022 and comprise 69,591,759 points with a resolution of approximately 51.05 points per square meter. In order to use the dataset during the thesis experiments, it was necessary to divide it into three parts comprising the training (70%), test (20%) and validation (10%) subsets and also to manually label the test and validation to be able to obtain quantitative evaluations of the different models. Initially, the current state of the art in this field was evaluated by collecting public datasets and analysing the best architectures used in this area. With regard to datasets, those existing in the literature and presenting a taxonomy similar to that of Turin were selected, i.e. Sensat Urban, DELFT SUM, ECLAIR, FRACTAL, Toronto-3D, STPLS3D, Swiss3D and Hessigheim. Next, several neural network architectures that emerged from the literature as the most promising in 3D semantic segmentation tasks were considered, i.e. RandLA-Net, Point Transformer, KPConv and SPConv. In the course of the research,several techniques were employed to improve the segmentation results on Turin. First, it started with the transfer learning technique, using for the training phase the concatenation of the previously selected datasets and employing all the different neural network architectures. After that, these models were tested on the Turin dataset to identify the best performing architecture, subsequent experiments, in fact, only focused on the one that had performed best in this initial step. Subsequently, self-supervised techniques were employed, using the confidence score of the best performing model's predictions on the Turin dataset as pseudo-labels during training. Several variants of these experiments were conducted to obtain the most accurate model possible on this new dataset.