Road Elements Identification and LiDAR Integration for Advanced Driver Assistance Systems

The fusion of multiple sensors' data in real-time is a crucial process for autonomous and assisted driving, where high level controllers need classification of objects in the surroundings and estimation of relative positions. This paper presents an open-source framework to estimate the distance between a vehicle equipped with sensors and different road objects on its path using the fusion of data from cameras, radars and LiDARs. The target application is an advanced driving assistance system which benefits the integration of the sensors' attributes to plan the vehicle speed according to real-time road occupation and distance from obstacles. Based on geometrical projection, a low-level sensor fusion approach is proposed to map 3D point clouds into 2D camera images. The fusion information is used to estimate the distance of objects detected and labelled by a Yolov7 detector. The open-source pipeline implemented in ROS consists of a sensors' calibration method, a Yolov7 detector, LiDAR points down-sampling and clustering, and finally a 3D to 2D transformation between the reference frames. The goal of the pipeline is to perform data association and estimate the distance of the identified road objects. The accuracy and performance are evaluated in real-world scenarios with real sensors data. The pipeline running on an embedded Nvidia Jetson AGX achieves good accuracy on object identification and distance estimation running at 5Hz. The proposed framework, introduces a flexible and resource-efficient method for data association from common automotive sensors and proves to be a promising solution for enabling effective environment perception ability for assisted driving.