Multi-task visual transformers for damage detection and re-identification in the insurance domain

In the assurance field, being able to recognize the presence of damages in an object is a delicate task, that requires dedicated human resources in order to help customers while also preventing frauds. In this work, I present an innovative method for tackling this task using machine learning techniques such as Damage Detection and Image Re-Identification, specifically for the case of bike insurance. Given the general scarce availability of real images for this specific field, I also took advantage of purposely generated synthetic images in our training process. I performed a comprehensive study and research work on both the Damage Detection and Image Re-Identification fields, which led me in developing a pure Transformer-based Multi-Task network that is able to perform both tasks simultaneously while obtaining promising results in both, and demonstrating how both tasks can benefit from each other. More specifically, the goal of this network is to, given a query image of a bike, being able to simultaneously detect the presence of damages on the the object while also Re-Identifying its matching pair in a set of gallery images. To the best of my knowledge, this is also the first work to adopt a pure transformer both for the damage detection problem and for Multi-Task learning on purely visual tasks. The results are evaluated taking into consideration both the performance on re-identification using mAP, Accuracy for damage detection of synthetic images and MCC and ROC AUC for damage detection of Real images. The final results achieved on the best model obtained are thus a mAP of 85.9%, a an Accuracy on synthetic images of 86.8%, and on real images a MCC of 73.4% and a ROC AUC of 98.9%.