The growth in computation power and storage strongly reduced the time required for a machine learning model to be trained and allowed the development of more complex models that can address more sophisticated tasks. These improvements led in the last years to the introduction of AI systems into a growing number of applications. Some of them are also used in contexts that could lead to safety issue, where a wrong behaviour, in some cases, could even endanger the life of a person. In the thesis we tried to deepen the understanding of malicious attacks that can be carried out against machine learning based systems.