Leveraging Quantization and Approximate Computing to Enhance Adversarial Defense in Deep Neural Networks

Over the last few years, Convolutional Neural Networks (CNNs) and other deep neural network architectures have been used increasingly across multiple domains. Such as computer vision, autonomous driving, and medicine. This widespread usage of CNNs has exposed them to adversarial attacks: applying deliberate perturbation to input data with the goal of forcing the CNN to produce wrong results. Quantization and Approximate Computing (AC) were originally introduced to reduce CNNs’ memory and computational cost. Furthermore, recent works have demonstrated that the noise they introduce could enhance input features, thereby reducing the likelihood of the adversarial fooling the CNN. In this study, we explore the effect of quantization and AC on the robustness of CNNs. We propose a software framework to train and evaluate quantized CNNs with support for layerwise approximation. Moreover, the framework provides adversarial data generation for various attack types, in addition to Quantization Aware Training (QAT), and adversarial training, allowing for extensive exploration. A multiplier architecture with 256 approximation levels is chosen and integrated into the framework using Look Up Tables (LUTs). Considering layerwise configuration with 256 levels available for selection, exhaustive evaluation of approximate level configurations is infeasible. Therefore, the genetic algorithm NSGA-II is used to find optimal configurations by maximizing adversarial and standard accuracy. Quantized CNNs led to an increase in adversarial accuracy of around 50% for a black-box attack and around 30% for a white-box attack, depending on the attack type and the chosen CNN architecture. For ResNet-32, AC led to a further increase in adversarial accuracy by 2-6%. However, this came at a cost: a 2% increase in adversarial accuracy had no effect on standard accuracy, whereas a 6% increase resulted in a 2% drop in standard accuracy. These results show that quantization effectively defends against adversarial attacks by significantly enhancing CNN robustness. While approximate computing offers only a modest improvement, it does not act antagonistically to quantization.