The widespread use of artificial intelligence (AI)-based systems brings with it several questions about the deployment of such systems in safety-critical contexts. Several industry standards exist, such as ISO26262 for automotive, that require detecting hardware faults during the mission of the device. Similarly, new standards are being released concerning the functional safety of AI systems (e.g., ISO/IEC CD TR 5469). Hardware solutions have been proposed for the in-field testing of the hardware executing AI applications, but when used in conjunction with complex applications such as Convolutional Neural Networks (CNNs) in image processing tasks, they may increase the hardware cost and affect the application performances.