AI-driven Computer Vision Design and Implementation for a Grape Harvesting Robot

This master’s thesis presents the design and development of an autonomous grape-harvesting robot, specifically engineered for operation in a real vineyard. The primary aim is to develop a robot that can accurately and gently harvest grapes, considering their delicate nature and need for a careful touch. The system consists of a 3-degree-of-freedom (DOF) robotic arm that positions an end effector near each grape cluster. The arm’s movements are controlled by an STM32F401RE microcontroller, which directs the motors and drivers to ensure precise placement. A key focus of the research is achieving real-time vision processing, facilitated by the robot’s central unit, an Nvidia Jetson Orin Nano board. This board was chosen for its optimized computer vision capabilities, managing both the process state machine and neural network inference to detect objects, such as grape clusters, through a connected camera. A YOLOv8 neural network, trained on over 300 vineyard images, was selected to identify and locate grape clusters, enabling robust detection even in previously unseen conditions. This computer vision approach has been tested in controlled environments, demonstrating the robot’s ability to efficiently identify and approach grapes without causing damage. For the delicate grasp of the grapes, a custom-designed robotic hand is equipped with fingers that close gently around each cluster. This hand operates via a custom PCB with integrated pressure sensors providing real-time feedback to prevent excessive force. The final harvesting step involves a cutting mechanism within the hand, which severs the grape stem after the neural network accurately identifies the cutting point.