Safe Inspection of Harsh Environments Enhanced by Virtual Reality

The Electric Arc Furnace (EAF) inspection is a common activity in steel making industry. Currently, it is done visually by the operator. This type of inspection cannot map the entire EAF surface. The operator can see only a small part of the furnace through an opening used to remove the slag. Moreover, this inspection is not precise, as the operator can examine the furnace through the heat shield and only for a brief time. In addition, the visual inspection cannot find local temperature differences. Furthermore, these manual inspections are dangerous. The main goal of this project is to implement a robotic system able to make EAF safe inspections, focusing on the refractory layer and the water cooling plant. This system is composed of a robotic tool with Color and IR cameras to take photos from inside the furnace. Then, the images are sent to a remote server for the storage. The user can see the photos and interact with them (search for a photo, select it, make global and local notes) using an application software. In order to reduce the imaging system cost maintaining a low inspection time, the tool uses only one camera and a normal lens per type, in place of multiple cameras arranged to cover the whole horizontal FOV of nearly 360° or a camera with spherical lens. The cameras are mounted on a rotary table. This solution requires image stitching, in order to create a single panoramic image starting from the furnace photos. The work exposed concerns the design, implementation and testing of all the SW architecture needed to stitch the photos coming from the tool, send them to a remote server and allow the user to see and interact with them. Working with thermal cameras, the stitching algorithm is challenging: state of the art feature-based techniques are not reliable with IR image. Thermal photos are extrinsically homogeneous, so they are hard to tackle with these techniques. To solve the problem, a different stitching method is proposed, based on the camera pose knowledge. The Homography matrix is calculated from the rotating angle of the camera, measured by an encoder mounted on the rotary table shaft. The proposed algorithm has proved to be robust and reliable with the IR furnace images. Another problem encountered is the lack of an effective evaluation index for the stitching quality. For this reason, a new score definition is proposed, which overcomes the limits of the current measurement algorithms and can be applied to every kind of stitching algorithm. The stitching score is based on a grid of point applied to each input and output image. The score is than calculated, starting from a point score up to a global one. The score can detect both image misalignment and stitching artifacts. It can be applied to both images and videos. As regarding the User Interface, Virtual Reality is well suited for panoramic images, due to their wide FOV. In order to have a maintainable and usable system, a WebApp solution has been implemented. The user can interact with the images using a software application primarily designed for VR headsets, but accessible from every device equipped with a Web Browser. The software application implemented achieves all the project requirements. Nevertheless, room for improvement is present in both the VR and the image stitching parts. This system showed its potentialities during the test phase, when it discovers a real coolant loss, not revealed by the visual inspections.