Deep Learning-Based Depalletizer: Object Localization with Real and Synthetic Data

This thesis investigates the use of NVIDIA Isaac Sim, a robotics simulation platform built on NVIDIA Omniverse, designed to simulate and validate AI- driven robotic solutions within physically realistic virtual environments. The main objective is to assess the effectiveness of synthetic datasets generated through Isaac Sim and their applicability in industrial scenarios such as depalletization and bin-picking. Synthetic data—artificially produced information obtained from virtual simu- lations—represents a strategic asset when real data is costly, difficult to acquire, or prone to bias. To this end, a test environment faithfully reproducing real oper- ating conditions was developed in Isaac Sim, introducing variability in position, orientation, lighting, textures, materials, and in the physical characteristics of boxes/objects and containers. The simulations generate multiple types of outputs (RGB images, instance/semantic segmentation maps, depth maps, and surface normals), which are subsequently employed to train modern segmentation networks (e.g., Mask2Former). In addition, two robotic hand-simulations: vacuum and parallel-jaw, were implemented to produce labeled results and gripper information, which also included unsuccessful attempts on non-graspable objects. Different methods were used to train the Mask2Former model: supervised on real, synthetic, and mixed datasets; domain adaptation; and synthetic-only (zero-shot sim-to-real). A GUI, API, and AI-assisted annotation tool based on SAM are developed to improve the efficiency of data generation and labeling. The findings demonstrate the potential of synthetic data and domain adaptation to reduce the sim-to-real gap for industrial vision tasks.