AUTOMATED DATA ANALYSIS AND FUSION APPROACH FOR AUTONOMOUS VEHICLES IN MIXED URBAN TRAFFIC

The increasing demand for autonomous vehicles (AVs) in transportation services has prompted authorities to optimize traffic performance, reduce energy costs, enhance drivers' safety, and detect traffic violations. Although AVs are equipped with various sensors such as Lidar, speed, lights, etc, in order to analyze the interactions in a mixed traffic flow, and to keep track of the unusual patterns observed in received sensor data, analysis solely on the sensor-based dataset is not enough. In this thesis, this challenge is tackled by leveraging fused information coming from two distinct sources: sensors implemented on the shuttles (AVs) and cameras implemented alongside the road. The main focus lies in minimizing human inference and so automatizing the process of anomaly detection and comprehension, monitoring and understanding the reasons behind in order to reach the comprehensive analysis in urban mobility where all AVs, traditional vehicles, and pedestrians exist. Initially, following the completion of pre-processing, visualization, and anomaly detection steps on the sensor dataset, the process proceeds to address the phases pertinent to camera recordings. These encompass camera calibration, object detection and tracking, pose estimation, and spatial data analysis. The objective is to seamlessly link both data sources leveraging these phases when anomalous behavior is observed in the sensor data. The proposed algorithm yields accurate results demonstrating the efficacy of this approach in identifying anomalies, understanding the distance-based reasons behind, and providing valuable insights to reach the comprehensive analysis in urban mobility where AVs exist. This work underscores the importance of automated methods in improving autonomous mobility in urban traffic, with implications extending to various stakeholders in transportation and beyond.