Detection and Recovery of Errors in Object Detection with Faulty Convolutional Neural Network

Convolutional neural networks (CNN) are extensively used in computer vision tasks such as image classification and object detection. Depending on the field of use, the decision-making process based on a CNN may be a sensitive issue because it affects the behavior of a system that works in critical applications such as autonomous driving systems. For that reason, anomalies in the CNN estimations have to be confined. This thesis proposes a solution to detect and recover errors in a CNN's output. The errors can be of two types: miss detection and wrong detection. The proposed algorithm leverages the temporal correlation among consecutive CNN outputs to generate interesting features that suggest whether an error occurs. Miss detections are solved by recovering the lost objects with prediction algorithms, whereas wrong detections are solved by canceling non-existent objects with a filter. The proposed method has been implemented as a CNN-agnostic algorithm called Bounding Box Recovery (BB-Rec) that uses the same inputs and outputs processed and generated by the CNN. In order to evaluate the performance, precision and recall of the stand-alone CNN and of the same CNN supported by BB-Rec have been computed and compared using MOTChallange dataset and py-motmetrics benchmark. Performance gains are achieved by the supported-CNN that presents, as the main result, improvements in recall combined, however, with a decrease in precision. These results show that the proposed approach can effectively enhance CNN's recall metric, recovering lost bounding boxes.