Development of a Human-Centered Framework for Safe Object Handling with Mobile Manipulators

In recent years, the use of robots has become increasingly common in people's daily lives. The ability to delegate simple tasks, such as floor cleaning or lawn mowing, simplifies and enhances the quality of life. The fundamental idea behind this thesis is based on this premise. The project focuses on developing an assistant robot that can coexist in a domestic environment alongside people. For the development of this thesis, a mobile manipulator was used, programmed to: (i) receive user requests, (ii) locate itself in a known closed environment and navigate safely in presence of people, (iii) identify the object requested by the user, grasp it and place it in a predefined position, and (iv) deliver the object to the user safely. The idea is to create a robot that can assist people with mobility difficulties, including disabled and elderly individuals. This robot will be utilized in indoor domestic settings. However, its capabilities can also be adapted for use in industrial settings, where it could serve as an assistant for workers. The development of the robot is designed to be versatile, thus facilitating its use in different contexts. In order to achieve such versatility and effectiveness in a variety of settings, recent advances in artificial intelligence and machine learning have been employed. The robot processes data in order to determine its actions in real time. In this case, neural networks from Ultralytics were employed, specifically YOLOv8 for object detection and YOLOv8 Segmentation for image segmentation. The networks were used for the purposes of tracking individuals during the navigation process and for the computation of the grasping pose of the requested object. Additionally, MediaPipe's Gesture Recognition was used to develop a safe object handover maneuver. The software architecture of the system was developed using ROS. An Android app was developed to facilitate human-robot communication. The robot lacks an integrated microphone, so the user-friendly nature of a smartphone or tablet was employed in the development of software capable of communicating directly with the robot. Each developed algorithm and strategy was tested on the Locobot wx250 in simulation and laboratory.