Autonomous mission configuration on Spot from Boston Dynamics

Nowadays, robotic technologies are used in many industrial sectors as they provide huge benefits to companies and workers; for example, in industrial environments, inspection tasks have been increasingly automated since they are jobs susceptible to human errors. In this context, the use of robots allows to get more precise results and to deploy the human workforce in non-automatable activities from which the companies can benefit. Autonomous inspections have an important role in this thesis work which is part of the project carried out by Sprint Reply, the society of Reply group specialized in hyperautomation with focus on Robotic Process Automation, Computer vision & ICR, AI & Machine learning and Process mining. The project aims to develop a robotic solution for a world leader company in the energy industry. The objective is to detect and monitor the conditions of working machines by reading meters located in one of the company’s plants. To this purpose Spot, the quadruped canine-inspired robot from Boston Dynamics, was employed. This robot is equipped with cameras and sensors, it has great stability and ability to adapt autonomously to height differences in the terrain; also, payloads can be mounted on top of it to improve its performances. These features allow Spot to automate inspection tasks and to monitor different scenarios, detecting and recognizing specific objects and capturing data safely, accurately, and frequently. The project was divided into two phases. The first phase included data collection, gathering and labelling, model training exploiting machine learning techniques, computer vision algorithms development and their packaging in Docker containers. The objective was to collect photos of the target meters located in the customer’s plant, label them, and employ them to train the model: this makes the model able to recognize the objects to detect. In addition, computer vision algorithms have been developed to extract, characterize, and interpret information coming from the collected images. Some examples of the meters of interest are analog gauges, mechanical and digital water meters, digital indicators, expansion vessels, status indicator valves and liquid level gauges. The second phase of the project was held on-site and involved mission configuration on Spot, testing, validation, and analysis of the results. Three inspection tours have been identified and, for each of them, the number of checkpoints that Spot had to take has been defined. Then, mission configuration on Spot started. It involves two steps: mission recording and mission replay. Mission recording is the process by which the robot is instructed on the path to take and the actions to perform. One mission was recorded for each inspection tour and subsequently it was replayed periodically. This started the test phase in which the computer vision algorithms were revised to achieve the final goal which is to have AI reading reach ever higher success rates. For the validation of the results, a comparison between the human and AI-based readings have been made. Specifically, the result coming out from the AI-based reading needed to differ by less than a certain tolerance with the number read by human eye on the meters. From the results, it emerges that the AI-based reading reached on average a success rate of 70% which is an acceptable value for the limited time available. Finally, the results were illustrated in one of the company’s steering committee meetings which resulted in positive feedback.