Transferability of Adversarial Attacks: Main Influencing Factors

The growth in computation power and storage strongly reduced the time required for a machine learning model to be trained and allowed the development of more complex models that can address more sophisticated tasks. These improvements led in the last years to the introduction of AI systems into a growing number of applications. Some of them are also used in contexts that could lead to safety issue, where a wrong behaviour, in some cases, could even endanger the life of a person. In the thesis we tried to deepen the understanding of malicious attacks that can be carried out against machine learning based systems. The most common type of this attack is known as adversarial examples. It allows to fool a network into a wrong behaviour by providing slightly ad hoc modified inputs. The relevance of this attack is due to the fact a human observer cannot notice the difference from a normal input. In detail, we analyzed the property of transferability of an attack by focusing on the knowledge the attacker has with respect to the target ML system. In this context, we analyse how knowing the training set and the network model affects the transferability of an adversarial example. The research analyzes both image and face recognition challenges, with the application of more complex netowrk models to address the second problem. The results highlight that the knowledge of the training set is relevant only in the case where also the network model is known. Otherwise, knowing the dataset only affects the result of few percentage points. What happen to be significant in the generation of the adversarial example is the quantity of noise we introduce. Moreover, comparing the scenario of image recognition with the one of face recognition, we notice a remarkable drop in the number of transferable case and in the influence of the quantity of noise. We attribute this difference to the higher complexity of the network used in the second challenge.