Deep Learning on the Edge: a comparative analysis on Computer Vision for space applications

Artificial intelligence is currently one of the topics of most interest in computer science. In particular, neural networks have found applications in many disciplines, like autonomous driving systems, medical diagnosis and many others. Enabling technology for this IT revolution is the available computing capacity provided by the Cloud. In some scenarios, however, execution on external servers is not possible, as in the case of some space missions. The only solution is to run the algorithms directly on the device that generates data. It is, therefore, necessary to migrate system intelligence from the Cloud to "the Edge". To do so we must optimize the models to minimize the computation and memory required. This paper will show the whole development process, starting from network training, through model optimization to deployment on embedded devices. For our tests, we employed the SPEED Dataset and trained three different neural networks for typical tasks in computer vision, such as object detection, semantic segmentation and keypoints detection. The models are optimized following standard techniques, already present in the literature like quantization. The devices where we intend to run the algorithms represent the most popular off-the-shelf solutions currently on the market, such as Intel Neural Compute Stick 2, Google Coral Dev Board, Raspberry Pi, Nvidia Jetson Nano and Xilinx FPGA. We will analyze the accelerators equipped on each board and report a comparative analysis on various aspects of the development frameworks. Our results will mainly focus on accuracy, model size, and inference time. We will show how these solutions can effectively enable real-time machine learning applications at the edge. Although each device has its pros and cons, and we cannot say in general which one performs better, with the Coral Dev Board we get the best results. We can speed up about 80 times the same model running on Raspberry Pi 3B+, obtaining inference for semantic segmentation at more than 30 FPS.