Deep learning computer vision algorithms for apple localization and tracking - Simulation, implementation and validation

The world population is growing faster than ever, with an expectation of 10 billion inhabitants on the planet by 2050. To sustain such rapid growth, the agri-food sector must necessarily improve its production capacity and its efficiency, innovating through technological drivers that aim at optimizing and automating the process, while at the same time embracing approaches that can guarantee the sustainability of the supply chain. In particular, automated and precision agriculture is spreading across the industry, both through fully automated machines and through cooperative robots. Computer vision is the main enabler of this industrial shift, thanks to the enormous improvements the field has experienced through Machine Learning and Deep Learning. These technologies radically changed the way the tracking and detection problems are approached, making real-world applications much more convenient and effective. The main industrial applications revolve around localization of crop fruits, assessing of its maturity state and its agricultural needs, building a 3d map of the orchard and being able to navigate autonomously through real-time mapping of the surrounding environment. For every of the listed applications, the foundation of the system is a reliable model that is able to detect and track an object consistently. The best solutions known in the literature to accomplish such a task are Mask-R CNN, SSD and YOLO for the detection part and SORT and DeepSORT for tracking. This thesis aims to study a system capable of detecting and counting the fruits of a crop through an implementation of Y.O.L.O. V4 combined with a SORT tracker. The main goal of the system is to be reliable, fast and flexible. While the system has been tested on the specific case of apple orchards, it is able to perform counts for any type of fruit. Several steps have been taken to develop and validate the system. First, a simulation environment was built to correctly validate the system results in the detection, tracking and counting parts, in order to have a reliable reference. This environment was developed with real-world counterparts as a reference, also implementing eight different light conditions representing eight different hours of the day. Second, the performance of the system has been validated in real-world scenarios through videos under different conditions. The last part presents a re-evaluation of the previous work, both virtual and real, by retraining the YOLO model with a dataset provided by PIC4SeR (Politecnico of Turin Interdepartmental Centre for Service Robotics) that features apple orchards located in the countryside near Cuneo. The resulting system shows an average counting error spanning from 7% to 13% both in the simulated and in the real environments, with sensitivity on the measure due to light conditions. After the retraining, the counting error outputs minor improvements in the simulations, while in the real applications it is seeing counting errors spanning from 11% to 6%. This work can be taken as a basis for future developments of more advanced systems, capable of carrying out automatic harvesting, perform crop mapping operations or assess fruit maturity state.