Vision detection algorithms for a robotic bartender

Vision systems have emerged as innovative and powerful tools thanks to their remarkable versatility, adaptability to diverse scenarios and good performance even with cheap hardware configurations. This thesis presents a practical example of application of these innovative systems, which is the integration of a vision sensor into the sensing set of a robotic bartender to “upgrade” its detection capabilities. The analysed machine is “Compatto”, which is one of the latest releases in the field of fully automated systems for bartending and has been developed by the Turin-based company MakrShakr. The idea underlying the dissertation is that it is possible to substitute the photoelectric sensors with a single camera, to obtain a great increase in the flexibility of the system, lowering at the same time the machine production costs, without losing any efficiency. The study delves into the development and integration of vision detection algorithms, comparing their performances and analysing their working principles. Three main approaches are investigated: a naïve one, based on a threshold technique aimed to the shape matching; a feature matching approach, based on Haar-like feature; and neural networks. All these techniques have been tested on the same hardware set up, composed by a small USB camera module, connected to a Raspberry Pi 4 model B, which communicate directly with the PLC of the robotic bartender through ethernet protocol. Furthermore, thanks to the adaptability of the developed detection algorithms, in the paper it is also proposed an efficient and reliable method to translate the high-level information coming from the camera sensor into Booleans, which are able to control the motion of the robotic arm during its missions. On the whole, the high level of data acquisition generates discussions about potential future enhancements to the robotic system's features. To summarize, it is possible to say that the study provides insights into the fusion of robotics and image analysis technologies and explores the potential of computer vision, begin it a feature which can revolutionize automated service industries.