Hardware Convolutional Neural Network based on Residue Number System

Hardware Convolutional Neural Network based on Residue Number System The prevailing numeric system in arithmetic circuits hardware implementations is unquestionably the Binary Number System (BNS). Nonetheless, over the last few years, an ever-greater shift towards parallel architectures is highlighting the limits of this arithmetic representation. In particular, being it a positional representation, the carry chains processing time constitutes the strongest limitation to the parallelism exploitation. Representing numbers with unconventional number systems seems to be one of the most promising options to overcome this limit. Among the unconventional number systems proposed in literature, the Residue Number System (RNS) is one of the representations with the greatest potential. Based on the Chinese Residue Theorem, in the RNS an integer number is represented with a unique set of smaller co-prime numbers. The power of this representation is that it allows to parallelize the operations of sum, subtraction and multiplication: in fact, instead of performing a single operation on large integers as in the BNS, using the RNS representation it is possible to execute several parallel and independent operations on smaller integers, reducing the overall operation latency. In this master thesis work, the efficiency of this unconventional number system has been investigated when used to implement a Convolutional Neural Network (CNN), that is a well-known computing system which has already proved prone to parallelization and that makes wide use of additions and multiplications. The first part of the thesis work consists in the design and training a CNN for real-time handwritten digit recognition. The design and training flow is described in the thesis, together with an overview of the code used to implement it. The code has been written in the Python programming language, and it consists in four main elements: the training environment, that allows to train the network by adjusting its parameters using the back-propagation method, the test environment, that allows to evaluate the accuracy of the network by executing several predictions using a reference data-set, the floating-point model of the CNN, where the layers are implemented in floating-point precision both in the forward and backward directions, and the fixed-point model, used to evaluate the actual accuracy of the hardware implementation for different fixed-point bit-widths. The thesis work continues with the presentation of the CNN hardware implementations. Except for the convolutional layer, all the layers of the CNN have been implemented using the BNS representation, since they are not critical in terms of computation time and they contain operations that are not efficient to be implemented in RNS, like the comparison operation. The thesis work focuses on the convolutional layer, that is the most critical layer of the CNN and mostly consists of additions and multiplications. It has been implemented using the BNS and the RNS representations, in a parallel and a serial architecture, and using four different fixed-point precisions, 16-, 22-, 26-, and 32-bits, for a total of sixteen implementations of the same CNN. Lastly, the thesis results are presented: BNS and RNS implementations are compared in terms of performance, area and power consumption, showing how the RNS-based implementations can effectively lead to better results respect to the equivalent BNS versions.