DA-PanopticFPN: a panoptic segmentation model to bridge the gap between simulated and real autonomous driving perception data =

The scene understanding component is a fundamental part of autonomous vehicle systems: it provides the necessary inputs to the downstream planning and control algorithms required to navigate the real world. In such context, to provide the necessary robustness guarantees of the deep learning models employed in the perception system, simulation is often resorted to as a means to perfect such algorithms in complex or edge case driving scenarios. However, perception models trained on simulated data and tested on real environments are affected by a domain shift, which often hinders the improvements obtained by the use of simulation. Hence, this work focuses on addressing the domain adaptation problem for the novel panoptic segmentation task, in the context of autonomous driving perception. To achieve this objective, a tool to generate synthetic auto-annotated panoptic data has been developed, by means of the CARLA simulator. Then, the resulting synthetic large scale dataset has been utilized with a reduced set of semantic categories so as to match those of the real world datasets cityscapes and BDD100k. Consequently, this joint dataset has been utilized in the unsupervised adversarial domain adaptation framework, to train and validate a domain adapted version of the PanopticFPN panoptic segmentation model, here named DA-PanopticFPN. This work shows that such technique allows to obtain a 10% and 19% improvement in panoptic quality on the bdd100k and cityscapes panoptic test dataset, respectively, compared to a PanopticFPN baseline model trained solely on simulated data and tested on the real scenarios.