FPGA implementation of a vehicles detector

The growing demand for self-driving vehicles in the market leads to investigate new optimized and higher performance technologies. Obstacle detection on the road is one of the key tasks for this purpose and it requires very short response times to avoid any possible risk. This work has the main objective to develop an AI model for FPGA-based devices. YOLOv5 is one of the most efficient models to satisfy the object detection task. The expected final pipeline is made of an input sensor to acquire the traffic images and an output display to visualize them with the vehicles surrounded by rectangles generated by the inference process. This last operation is performed by the FPGA-based system that must be properly built by software and hardware perspectives. The whole design is supported by the AMD software tools, such as Vitis AI, Vivado, Vitis and PetaLinux. While the target FPGA-based board is part of the AMD ones and it is known as Kria KV260 Vision AI Starter Kit. Initially, a theoretical background is shown about Artificial Intelligence and Autonomous Driving with an additional look at the hardware platforms able to run AI models. Moreover the AI model preparation is discussed: the YOLOv5 neural network architecture, the training process together with the used dataset (Kitti), the analysed metrics and the validation of the final trained results. All this concepts are a strong base for the next description that can easily divided into three macro-steps. Firstly the Model Development phase will lead to a working compiled model. The Vitis AI environment with its main tools is presented. The quantization converts the 32-bit floating point model into a 8-bit integer one and, then, the compilation translates the model into the XIR-based format. This means having a deployable model, since its format is readable by the board’s DPU, i.e. the base IP of the FPGA-based AMD devices. After a presentation about the entire pipeline, from the frames acquisition sensor to the visualization of the inference results, the Platform Development step is analysed. The FPGA-device boards typically need of a platform that enable all the necessary hardware elements available on the board. The advantage of the Kria KV260 board’s use is that many ready applications are available and open-source. They gives the possibility to use their Vivado and Vitis base scripts as starting point for this thesis design. Vivado is used to build the Xilinx platform and Vitis to build the overlay and join together in a single xclbin file these two components. All these firmware elements are then loaded into a Linux image, properly built by the AMD PetaLinux tool. Ready the hardware side of this design, the Application Development is the next step to directly run all the operations: pre-processing, inference and output display. It consists of easily writing an application code with the support of the GStreamer framework, a series of plug-ins to manage the multimedia pipelines as this one. Finally, all the elements, i.e. the Linux image with platform firmware and the model with all the supporting configuration files, are deployed on the Kria KV260 board that, at this point, will be able to detect the vehicles on the input frames. The several design layers, carried out in the whole thesis work, introduce critical issues that can compromise the final working. The choice of higher performance training machine, better quantization approaches and less corrupting pre-processing lead to more optimized results.