Sensor Fusion: Computation of a Transformation Matrix between a LiDAR and a Camera

The purpose of this work is to provide an overview of the sensor fusion process involving LiDAR and stereo camera sensors, and to show which are the steps needed to compute the transformation matrix that allows the communication between the two instruments. In order to understand a bit more the sensor fusion’s world, a brief introduction on the topic is presented, followed by a more in-depth analysis on the two sensors used for this project, described in order to understand how they work, which are their strengths and which are their weaknesses. A series of some of the most distinguished methods of extrinsic calibration between LiDAR and camera, is then shown before proceeding with the presentation of the method that we decided to employ to solve that same task. In particular, it is presented the MATLAB program used in order to compute the matrix and the Python code built in order to verify that the estimated transformation matrix worked correctly also outside of a MATLAB environment. Lastly, along the obtained results, are discussed what might be the future developments for this work and how the focus is the real time implementation for obstacle detection.