Design and Implementation of a Robotic End-Effector and State-Machine Algorithm for Autonomous Grape Harvesting System

This master’s thesis presents the design of a robotic hand aimed at autonomously harvesting grapes for wine production, terminating in a functional prototype. The primary objective is to achieve human-like arm and hand movement, facilitating safe interaction with delicate fruit and reducing the risk of damage. The first phase of this project involved designing custom hardware for a PCB to monitor, in real-time, pressure sensors attached to the robotic hand and controls motors connected to the fingers. This custom board is optimized to measure the pressure applied, emulating a human sense of touch, and adjusting directly the motors response. Bluetooth communication ensures an efficient method to receive commands from the main computational unit and coordination with other custom boards. Then, the robot hand's end-effector was re-engineered to primarily reduce its overall weight and dimension. Specifically, the palm of the updated end-effector was developed in this thesis to integrate stronger motors, custom boards and pressure sensors. The kinematics of this robotic hand is based on an open-source prototype, with a focus on maximizing hardware space while ensuring an efficient grape grasp and maintaining flexibility and range of motion. This compact build ensures operations in a narrow space commonly encountered in a vineyard. The final phase involved high-level software development to coordinate and control the robotic hand’s operations. A Pyhton-based state machine algorithm was implemented to manage the sequence of movements. This algorithm was developed to ensure smooth and reliable transitions between states, such as identifying, reaching, grasping, holding and depositing grapes in a designated collection area. The software runs on a Jetson Orin Nano, which also hosts a computer vision model to recognize grape position in real-time. This advanced visual recognition enables the hand to orient itself correctly, even in dynamic environments. Additionally, the Jetson communicates with a Nucleo STM32F401RE via serial communication to control the motors at the base of the arm, coordinating overall movement. Meanwhile, Bluetooth communication with the custom boards allows for controlling the remodelled end-effector and the cutter arm. This modular approach permits easier management and scalability, enhancing the system’s flexibility. Tests and validation were carried out in multiple stages. Initially, the boards were assessed for any hardware defects, sensor accuracy, data processing and reliability of the Bluetooth channel. Following this, the redesigned hand’s structural integrity and mechanical reliability were tested with the upgraded motors and custom boards. The final phase evaluated the complete system's performance in harvesting task, confirming that the robotic hand accurately executed each discrete state. Although testing could not be conducted directly in a real-world vineyard, an artificial environment was used to simulate real-world conditions.