Increasing the abilities of mobile robots with Computer Vision algorithms

In the field of robotics, there is a growing trend towards the automation of mechanisms and tasks, reflecting a dynamic shift towards increased efficiency and productivity, with artificial intelligence playing a crucial role. This thesis project explores various use cases and projects conducted on Boston Dynamics' four-legged robot Spot, focusing on the integration of computer vision algorithms, enabling Spot to achieve remarkable level of self-awareness of the environment and improving its capacity to collaborate with humans. This thesis presents different neural networks aimed at enhancing Spot capabilities in industrial context, where it is thought to be used. Firstly, a face recognition algorithm is used to detect faces, which are then anonymized to protect workers' privacy. Secondly, another computer vision algorithm is employed to identify safety exits and fire extinguishers, ensuring workplace safety in industrial settings. Lastly, a third neural network is implemented to recognize the 80 classes from the COCO dataset, further improving Spot's surroundings awareness. These neural networks are integrated into a project that takes as input the streaming from the robot’s cameras, depending on the selected network, the system displays the inference results to the operator. Such versatility can be useful in many situations, as the robot is capable of replacing humans in repetitive inspections, which could lead to errors because of the lack of attention. Another relevant use case examined is the cooperation between robots. A first Spot, equipped with PTZ 30x zoom camera, is sent on an autonomous mission, which is a pre-recorded path with saved reference locations (waypoints) where Spot perform a proper action. The goal is to identify potential leakages within an industrial plant, an example could be leakage detection of a valve in a waypoint located near a tap. Collected data, as photos and videos, are transmitted to a cloud platform, where computer vision algorithms provide predictions, reporting alerts to an operator in case of faults. A second Spot, equipped with a robotic arm, receives a trigger from the cloud, assigning a specific mission on the base of the detected fault. In case of faulty tap, its task is to find the path to reach the valve and to close it. In this project is deeply covered the structural logic and the creation process of the second mission, starting from the establishment of a pipeline, which generates a synthetic dataset using photos of valves, passing through the training of the neural network and arriving to the setup of the mission, allowing Spot to autonomously navigate to the valve, identify the position, grasp and close it. This thesis work is born through the collaboration between Politecnico di Torino and Sprint Reply, all these research activities take place in Reply laboratory “Area42”.