Development and testing of an industrial vision application for robot guidance using OpenCV library

The purpose of this thesis work is the development of an open-source industrial vision application for robot guidance, implemented in Python language using Open CV library. The developed program aims to be a free-of-license alternative to the industrial machine Supata®, entirely designed, realized, and programmed by E.P.F. Elettrotecnica. It consists of a vibrating platform (the Supata® itself), a camera with a light source, and a robot, to compose a smart feeder system that singularizes parts randomly loaded into it and prepares them for following stations. This work covers all the development phases of this application, from the initial camera configuration to the achievement of the final output: detect pieces that can be picked, their orientation and grip point's cartesian coordinates, and, finally, the density distribution on the platform. The procedure starts with the calibration of the camera, necessary to eliminate the lens distortion effects, followed by the definition of the correlation between the 2D pixel coordinates of the image and the 3D millimetres coordinates of the robot. Once the camera is ready and the input image of the platform is acquired, the main program operates on two sides: a setup side, performed only the first time a new piece is considered, and a processing side, performed every time a new image is acquired. The setup side includes the definition of the master image and all the concerning parameters and features. On the processing side, after an initial selection based on the shape of the objects, the program compares every piece with the master, computing its orientation and the coordinates of the grip point. Then, after a final control to check the presence of obstacles in the grip area, the final list of pickable pieces is returned, as well as their orientation and grip point coordinates. Besides the coordinate computation, it also gives an information about the distribution of the remaining pieces on the vibrating platform. The comparison phase between a generic piece and the master is the core of this project and the most challenging aspect of the whole work, since it is the key to make this program applicable for every kind of piece, no matter the shape, the material, or the colour. To accomplish this task, a fundamental technique in computer vision known as feature matching is adopted, where the features of two images are detected, described, and matched by specific algorithms, implemented through dedicated OpenCV functions: in particular, the algorithms considered for this project are SIFT and ORB for the detection and the description of the features, and Brute-Force and FLANN for the matching. Finally, to test the program, a set of experiments has been prepared in which six different combinations of detector/descriptor and matcher algorithms have been applied on four pieces with very different physical characteristics: for each test it has been considered as quality evaluation criteria the number of well-matched and wrong-matched pieces, the computational time, and the precision with respect to the currently implemented industrial vision software on Supata®, based on Cognex libraries. The results of the tests show how the combination of ORB as detector and descriptor algorithm with FLANN plus homography as matcher algorithm has the best performance for every quality criterion, revealing itself as a promising starting point for future improvements.