Reinforcement Learning algorithm based on APF for Ground Robot Guidance

The increasing global demand for food, coupled with factors such as the shrinking agricultural workforce and the need for environmentally friendly practices has led to the emergence of Agriculture 4.0. In this context, Unmanned Ground Vehicles (UGVs) have become integral to smart farming, offering efficient and environmentally sustainable solutions compared to traditional machinery. To ensure the smooth and effective movement of UGVs, extensive research has been conducted on path planning algorithms. This thesis focuses on the development and implementation of a novel path planning algorithm specifically designed for point-to-point UGVs. This class of application is the most used in Precision Agriculture (PA). The algorithm is implemented and evaluated for two different application types: one where it serves as a guide for the UGV, and another where it functions as both guide and control. For the first case, an existing Sliding Mode Controller (SMC) is used. In particular, the developed method combines the Artificial Potential Field (APF) algorithm with the Deep Determine Policy Gradient (DDPG), respectively a classic path planning algorithm and a Reinforcement Learning (RL) one. By leveraging the strengths of both approaches, the algorithm overcomes the limitations of APF and RL, namely the local minima problem and generalization, respectively. The local minima problem arises when the attractive and repulsive forces generated by the goal and obstacles cancel each other out, leading to an undesired UGV stop. The algorithm addresses this issue by combining APF and DDPG, ensuring efficient and continuous navigation even in complex environments. Additionally, the algorithm ensures generalization techniques to enable the trained agent to adapt to different contexts and perform effectively in real-world scenarios. The agent has been implemented and trained in the MATLAB/Simulink environment, which facilitates the definition of the UGV’s physical model, the creation of various challenging training scenarios, and the creation of the artificial intelligence neural network through the use of specific toolboxes. Furthermore, the versatility of the MATLAB environment allows for easy code deployment in more practical and widely used languages like C++. This enables the possibility of translating everything into executable code supported by on-board systems. The numerical model on which the method was applied is a tracked UGV, which is a ground robot developed by the Department of Mechanical and Aerospace Engineering (DIMEAS) of Politecnico di Torino. To demonstrate the validity and effectiveness of the implemented method, numerous simulations were carried out in the MATLAB/Simulink environment. Furthermore, special scenarios have been created to validate the behavior of the algorithm in the presence of local minima. Finally, real applications have been performed on the UGV hardware platform to further validate the method.