A new benchmark for Anomaly Segmentation in driving scenes, using the CARLA simulator

Autonomous driving holds the keys to a future where driving time could be reallocated to other, more productive activities, where accidents due to human errors are minimized and where traffic could be optimized to reduce energy consumption. However, much progress still needs to be made for this to happen. In particular, handling of anomalous, hazardous objects, rarely seen by the vehicle during normal operation, is particularly difficult for the current state-of-the-art autonomous cars; this poses safety risks that sometimes lead to dangerous accidents, including pedestrian death. Furthermore, most deep learning models deployed on these cars are so called “black-box”, because it’s difficult to predict their behavior reliably and to identify what caused an accident after it happened. Research on these matters has been slow partly because it is difficult and expensive to produce datasets of road scenes with anomalies, making the development of deep-learning models to detect them difficult. One possibility to produce a high quantity of data with low economic cost is the use of graphic engines to simulate realistic environments with cities, vehicles and pedestrians. This thesis investigates the possibility of using CARLA (Car Learning to Act), a simulator based on Unreal Engine, to produce a dataset of road scenes filled with anomalous objects. The simulator, by offering software implementations of a variety of sensors like RGB cameras and LIDAR, often part of the perception modules of autonomous vehicles, offers also the possibility of recording different modalities of data. In the first part of this work a script is developed using the API provided by CARLA, that by taking user inputs and controlling the simulation as such, produces a dataset of road scenes in different maps and weather conditions. Then, after the collection of the dataset, some of the state-of-the-art models for anomaly segmentation on road scenes are tested on this dataset to assess their performance.