Reliability Evaluation of Split Computing Neural Networks

In the contemporary era, Artificial Intelligence (AI) has become integral to IoT systems, revolutionizing several fields. Due to resource constraints of these devices, various model optimization techniques are employed, such as split computing, where the workload is partly offloaded to the cloud to ensure that the required resources are within the capabilities of the employed devices. Despite these optimizations, models still require advanced hardware like GPUs which may be affected by faults. The graphics processing unit, or GPU, has emerged as a vital computing technology in both personal and business settings. It is specifically designed for parallel processing and is utilized in numerous applications such as graphics and video rendering. Nevertheless, GPUs are becoming more popular for use in creative production and artificial intelligence (AI), due to their capability to speed up computation, in case it involves simpler basic operations. The employment of such advanced hardware brings itself some risks particularly when they are used for safety-critical application. To assess the reliability of this sensitive hardware, Fault Injection simulations are carried out in order to find a relationship between the performance of the model and the features of its corruption. The evaluation of SC2 models reliability in the presence of GPU faults remains unexplained. This thesis work examines the impact of hardware failures on system reliability under the assumptions of the split computing approach that distributes neural network architecture between mobile devices and cloud systems in addition to a knowledge distillation process to maintain prediction accuracy and reduce transmission load that could be degraded by the injection of an artificial bottleneck. Furthermore, the research identifies vulnerable features through software-level simulations and investigates various hardening techniques on lightweight Deep Neural Network models. The findings exhibit substantial deterioration of accuracy in models subjected to fault injection during inference of several tasks, including image classification, object detection, and semantic segmentation. Suggested hardening techniques, including custom activation functions, show promise in improving model robustness and several simulation campaigns have been carried out and the corresponding statistics analyzed and compared.