Robotic arm pick-and-place tasks: Implementation and comparison of approaches with and without machine learning (deep reinforcement learning) techniques

A robotic arm is nothing more than a mechatronic structure inspired by the conformation of the human arm and capable of performing various tasks. The ability to perform tasks, in relation to the "hand" that is given, makes robotic arms very versatile; they can be programmed to perform any task a human arm can perform, however grasping objects is certainly the most interesting and requested. Pick-and-place in general is one of the most complete tasks that can be required of a robotic arm; although it may seem a trivial and immediate gesture for a man, for a robot it turns out to be a very complex task, it is not a coincidence that its complexity makes it a task well suited for studies and research. The intrinsic multidisciplinary nature of the field of robotics makes it suitable for the integration of techniques and knowledge from all parts of engineering and science, this leads to the introduction of one of the most profitable applications in this sector: the artificial intelligence (AI). The discussion focused on the concept of machine learning (ML): branch at the base of AI, union of several disciplines such as computer science and mathematics among all, which allows an entity to learn autonomously. The techniques for pick-and-place tasks have changed a lot over the decades, the aim of this thesis is to use a unified presentation of two approaches to deal with these tasks, first using the state of the art of the legacy of a so-called "classical methodology" (i.e. not including modern AI techniques), and secondly, instead, the same problem is addressed by treating the state of the art of the latest ML techniques for these types of actions. As regards the first approach, ROS framework with the MoveIt platform, Gazebo simulator and RViz visualizer were chosen, in order to implement the code and functionalities necessary both to perform a complete simulation of the arm and to allow integration on a real machine. Concerning the second approach instead, the reinforcement learning (RL) paradigm was exploited, specifically deep RL algorithms have been used because they are considered the most suitable and efficient based on research in the field; for this purpose, Gym (with MuJoCo simulator), Stable Baselines and RL Baselines Zoo toolkits were used to work with the "DDPG + HER" algorithm, which was in turn implemented through TensorFlow. The robotic arm used is that of the Fetch Robotics' Fetch robot. The author has chosen to organize the work on the basis of a rather widespread definition of the term "robotics", that is "intelligent connection between perception and action"; therefore, with this meaning, it is necessary that the robot perceives, reasons and acts. Precisely in this vision, the study and the two approaches refer to the part of the so-called "intelligent connection", specifically to the implementation of the reasoning entity: the motion planning system. The results of the thesis work are the implementation of the two robotic programming approaches described above and the comparison of their peculiar characteristics.