DEPTHWISE ADDERNET: Energy-efficient deep neural networks for edge devices

Today machine learning techniques and, in particular, deep neural networks are widely used for multiple tasks. Convolutional neural networks improve the performance of artificial intelligence algorithms in many applications (e.g. image recognition, object detection, natural language processing) but result to be energy-intensive due to the large amount of multiplications involved. This type of model is unsuitable for IoT devices or edge devices (such as smartphones) which are limited in terms of memory and computational capacity. For this reason, many studies are focusing on developing more efficient deep network architectures. This thesis analyses two modern techniques: depthwise separable convolution and Addernet.The first allows designing deep networks with fewer parameters while the second reduces the usage of hardware resources and decreases the computational and energy cost by substituting multiplications with additions. In particular a new layer is presented which combines the strengths of the two models. It is compatible with many networks in use today and it presents opportunities for further enhancements. Training and testing are performed on a GPU using the PyTorch framework. The DNN architectures employing the proposed layer present a reduction up to 75\% in the parameters memory occupation. Moreover when implementing the models on a dedicated hardware platform(e.g. FPGA) an improvement in terms of computational resource utilization and power consumption can be expected. Results obtained on different networks against CIFAR-10 and CIFAR-100 datasets shows the feasibility and potential of this new "depthwise adder kernel".