Segmentation-based approach for a heuristic grasping procedure in multi-object scenes.

The latest progresses in AI and robotics allow to automate many repetitive and tiring tasks. As a result, the focus of many activities can be transferred to more proactive and stimulating aims. The work is focused on automating piece picking, that is one of the most common tasks in the majority of industrial/logistic environments. As a matter of fact, depending on the available robot gripper (vacuum), an algorithm should be able to correctly identify suitable contact points from an RGB+D image. AI applied to image segmentation is a convenient way to achieve such a goal. At first, a solid model in charge of highlighting each class of objects in a cluttered scene is adopted, using a predefined dataset suiting this specific task. However, this kind of approach, namely multi-class semantic segmentation, is strictly related to the class labels of the chosen dataset. To increase the flexibility of this method, a change of perspective leads to a model able to segment unclassified objects, which is supposed to be more interesting from a practical point of view. This approach would not depend on the current dataset objects classes; hence it can be applied to detect the two generic classes, object and background, neglecting the semantic contents of each object in the scene. The first part of the new pipeline consists in adopting object detection models for bounding boxes prediction. The obtained detections are then fed to the state-of-the art promptable segmentation model, named SAM, which gives as final output the desired segmentation masks. The results are then compared with the ones of Mask R-CNN, a popular end-to-end learning instance segmentation model. Starting from the masks, proper grasping points are estimated by computing their corresponding centroid-like points. Such approach results to work well in most of the tested scenarios. KNN and PCA techniques are exploited to complete the objective pose of the suction end-effector. Both segmentation performance and final practical tests are reported to show the effectiveness of the entire work.