Object recognition and grasping with a UR robot

Visual servoing systems for the control of robotic manipulators are becoming a most relevant issue for robot application in unstructured environments. One of these operations is the capacity of a robotic arm to find the pose of an object and to use that information to pick it up. The goal of this thesis is the recognition of an object by an RGB-D camera image, and the use of a robotic arm to grasp it. To perceive that objective a Realsense D-435 camera and a UR5 robot were used. The RGB image coming from the camera was given as input for the Convolutional Neural Network (Mask R-CNN) utilized for the object recognition task, while the depth image together with a simple inverse projection transformation were used in order to find the X-Y-Z coordinates of the previously found object. This kind of approach was applied because it does not require a hardware with a huge computational power, can have a Network trained on different sets of objects, and it is possible to exploit it on camera models different from the Realsense D-435. An already trained Network was use. At first a simulation environment, called GazeboSim, was considered for preliminary test, and then real world experiments have been done.