Characterization of Software Libraries on Embedded Cores

Machine learning has been very successful in many applications. These applications range from image classification, machine translation, autonomous driving, virtual personal assistant to medical services. Deep neural networks are algorithms used in machine learning that runs data through multiple layers such as convolution, activation, pooling, and classification. There is a huge demand for deep learning applications on embedded devices. However, these layers, particularly the convolutions, require a massive amount of computation, which is one of primary consideration, especially for resource constraint devices. Due to the different hardware platforms and neural network libraries, it is essential to have a suitable methodology that could select the best neural network architecture for different application requirements. This thesis presents a methodology to predict the performance of neural network libraries on embedded cores in terms of runtime. By collecting runtime for different parameters for every kernel to constitute dataset and then using a machine learning approach to get a per-layer predictive model. The resulting predictive model tested against three well known embedded networks implemented with CMSIS neural network libraries on ARM m4 microcontroller. The per-layer predictive model presented excellent accuracy, percentage error between actual measurement runtime, and predicted runtime is under 3 percent. Our approach can be extended to other embedded hardware-software platforms.