Detecting the Unknown in Object Detection

Object detection is a widely explored task in the field of computer vision, whose main objective is to both correctly locate and categorize all the objects within an image, separating them from the background. However, traditional object detection approaches work under closed-set condition, which means that the model can only detect objects seen during the training phase. This represents a clear limitation as it is not suitable for practical use cases, where unknown objects will unavoidably appear at test-time and the model is com- pelled to predict them as one of the known classes. Therefore a more realistic scenario is represented by open-set in object de- tection, where images containing unknown items can be submitted during testing, requiring the model to detect them too. During recent years, vari- ous approaches have been proposed to break the closed-set assumption and make the model works in an open-set environment, however no standard has yet emerged about data to use during the training-test phase and open-set metrics for a fair evaluation of the model. This work seeks to fill this gap by defining a strategy of pseudo-labeling unknown objects at training time, allowing our model to learn from items that are not explicitely annotated in a self-supervised fashion. A set of metrics has been proposed in order to fairly evaluate the model’s performance under open-set condition. Experi- ments made on Pascal VOC and MS-COCO show that the proposed method achieves the new state of the art, with a significant margin over previous approaches.