Hardware architecture for optimized neural network-based inference on autonomous driving systems

The majority of automotive original equipment manufacturers (OEMs) are including one or more systems that assist the driving task by avoiding collisions, warning the presence of obstacles, detecting road signs, constantly monitoring the driver for fatigue detection, etc. All these system are collected under a category called Advanced Driver-Assistant Systems (ADAS) that builds the foundation for developing the full autonomous vehicle: a model of vehicle that is able to drive itself. Advancements in this domain are empowered by the use of neural networks, which have demonstrated unprecedented levels of accuracy compared to the standard computer vision algorithms used before. The computation intensity for running neural networks is high and thus, different hardware platforms have emerged on the market outperforming the classic GPUs used in precedence in terms of intensity and power consumption. This thesis, developed in partnership with Marelli, discusses possible hardware solutions that will facilitate advanced assistance functionalities to be present not only on high-end car, like usually appears to be the market today, but also on less expensive or low-end cars. Important factors such as: latency, throughput (FPS), accuracy level, reliability, cost, safety have to be considered carefully when making a choice. To draw a complete picture and approach this topic in a seamless way an introduction to the neural networks is provided with particular emphasis on Convolutional Neural Networks (CNNs), followed by some state-of-art topologies developed in recent years. The framework used in this thesis to develop models of neural networks is Tensorflow, that is introduced in section 2.4. Different types of hardware architectures for inference like: CPU, GPU, FPGA, DSP, Many-cores, ASICs and neuromorphic are examined and some possible optimizations adopted by hardware vendors are described. Next, the focus is brought to a particular platform provided by Texas Instruments called Jacinto 6 where the main SoC is from TDA2x family. The subsystems of TDA2x are described in detail, and the development flow adopted to deploy a neural network on hardware is described step-by-step. In the last chapter, some metrics to evaluate hardware accelerators for DNNs are listed along with benchmarking models used or proposed by different studies. The roofline model used mainly for multi core architectures is described, as well as some examples of well-known DNN accelerators adopting this models are demonstrated. Lastly, a benchmarking model is developed, applying varying complexity and memory requirements to spot some the hardware limits of TDA2x SoC.