Image Recognition Benchmark with Different Embedded Solutions: Google TPU, RockChip NPU, NVIDIA GPU

Internet of Things has become more and more popular in recent years. Smart devices can send messages through the network, transfer a large variety of data, improve the quality of various services and security. With the rise of edge computing, many different IoT systems have been developed, and we now have the possibility to use them to work with deep neural networks. To make proper use of these devices, we need to know what are their limits and possibilities, and because each system has its unique characteristics, what can help is a series of experiments that make them in comparison under the same circumstances, to understand what are the advantages and disadvantages for each configuration. This work is an image recognition benchmark on different edge devices: Raspberry Pi, Coral USB Accelerator, Coral Dev Board, Rock Pi, and NVIDIA Jetson Nano. With the help of a test dataset and a selection of convolutional neural networks, we created a framework to compare the performances in terms of accuracy, inference time, and power consumption. From the experiments, it emerged that the Coral Dev Board is the fastest, Jetson Nano achieved the highest accuracy, and Rock Pi is the system that consumes less power during inference. Finally, we used what we learned to train a neural network with a self-made dataset for a future deployment in one of the systems we analyzed.