Development of Computer Vision and Control Algorithm for real field application of 3 DOF harvesting robot for vineyards

This Master Thesis aims to develop a system able to detect in a correct and efficient way bunches of grapes and their location in space, and consequently, transmit data to a micro-controller, belonging to a VirtualLab board, to control a 3 degree of freedom robot such that it can perform harvesting of the bunches of grapes. Raspberry Pi 4B with a camera attached to it has been used to perform the computer vision object detection of the target and testing different convolutional neural networks in order to find the most suitable one in terms of performances and accuracy has become one of the most important aspects of the research, together with its training and improvement as well. Once achieved good object detection in terms of precision, to further boost the capabilities of the system, NCS2 (Neural Compute Stick) from Intel has been experimented and implemented allowing to reach values of frame per second up to 14 FPS and value of accuracy of the detection up to 86%. Overall, the system is able to perform the assigned task correctly, even if is worth saying that due to the fact that grapes are characterized by a very short harvesting period, we couldn’t do tests on vineyards and all the presented results have been obtained working in an artificial environment.