Mobile Manipulator as Smart Human Assistant for Safe Objects Handling

Nowadays robots, and in particular mobile manipulators, can assist humans to accomplish common tasks, behaving as smart assistants for the operators sharing the same workspace. Keeping this in mind, the objective of this thesis is to expand the capabilities of a mobile manipulator (Locobot WX250s) in order to make it able to: (i) localize itself in a partially known environment, (ii) avoid obstacles in it, (iii) recognize and estimate the pose of known objects, (iv) localize the most dangerous part of the requested item and (v) pick and place it in order to minimize the risk for humans. The place inside which the robot operates can be a Warehouse or a Laboratory, indoor environments where objects are stored on shelves, cabinets and drawers whose fixed positions can be considered as prior knowledge by the robot. The entire software infrastructure has been implemented on Ubuntu 20.04 using ROS Noetic as base to let the different modules communicate with each others. ROS is the most used middleware when it is necessary to develop any software for robots of any kind. It is full of packages implementing the most recent algorithms and it is also readily expandable to enrich the capabilities of the system. The SLAM task is achieved using the RTAB-Map package that, starting from the information coming from both the RGB-D camera and the Lidar mounted on the robot, is able to construct a Map together with two costmaps (Global and Local), used to move the mobile base avoiding undesired contacts. Obstacle avoidance is further expanded by building a 3D occupancy grid exploiting the Octomap abilities to reinterpret the point cloud coming from the camera. For what concerns the motion control, the mobile base is controlled using the Move Base package together with the Dijkstra's algorithm as global planner and the TEB local planner, while the arm 's controller has been designed using the MoveIt! framework and planners belonging to OMPL. In addition, a custom trajectory planner that consider the arm and the base together as a holistic system has been realized. This allow to better position and correct the robot's gripper placement when picking an object. The robot is also able to recognize and estimate the pose of known items thanks to the use of a YOLO v5 neural network. It has been trained to classify things belonging to a specific types list and to compute the bounding boxes to understand their space occupation. By fusing this information together with the depth data of the camera, it is possible to estimate the pose of the requested item. The network has been trained using a fully custom dataset built from scratches specifically for this purpose; it contains images of common objects that can be found in any laboratory. Pick-an-place operations are executed following a human-in-the-loop approach similar to what is done in Interactive Machine Learning algorithms. The robot recognize dangerous objects basing on their class and ask to the human a feedback about the most dangerous part. Once this latter section has been found, it grasps the object from there. To test the designed infrastructure, some experiments have been conducted in both a simulated environment, thanks to Gazebo and Rviz, and on the real Locobot WX250s inside the laboratory.