Transformer-based pre-trained models for Out-Of-Distribution detection

Standard closed-set deep learning approaches deployed in the real-world fail when test samples come from never-seen data distributions, which makes them untrustworthy for safety-critical applications such as autonomous driving or healthcare. A preferable behavior would be to raise an alert in case of unknown object categories, a task known as Out-Of-Distribution (OOD) detection. The common strategy to handle this task is to train (or at least fine-tune) the detector on the target data to make it learn the normality distribution, to then recognize test samples not belonging to it. Consequently, It would be necessary to gather a great amount of labeled target data and to perform the training procedure for each distinct downstream task. Those are restrictive characteristics for many real-world applications, due to data privacy rules, strict memory, and computational constraints (e.g edge computing). Two recently proposed strategies suggest tackling the problem by using only a pre-trained model: one relies on visual relational reasoning, while the other exploits vision and language. Both those methods focus on learning representations generic enough to enable the detection of out-of-distribution samples in multiple tasks, without the need for fine-tuning on the target data. This thesis presents a comparison of the two strategies and proposes a tailored transformer architecture that increases the explainability of the overall model.