Convolutional Neural Networks todays are widely used in many scenarios, thanks to their exceptional performance in terms of prediction accuracy. Despite they are considered a error-tolerant models, their deployment in safety-critical domains require a careful evaluation in order to foresee the effetcs due to system failures. Our goal is to test CNNs reliability during inference phase by highlighting when the classification results deviate from the correct one. In this work we propose a methodology that evaluates reliability through a fault-injection campaign. Permanent fault are injected at the output of multiplications between weights and input feature maps. Experimental results show that the network works properly only for faults restricted to certain locations, ensuring the same accuracy level of a fault-free inference.