Visual based local motion planner with Deep Reinforcement Learning

This thesis aims to develop an autonomous navigation system for indoor scenarios based on Deep Reinforcement Learning (DRL) technique. Autonomous navigation is a hot challenging task in the research area of robotics and control systems, which has been tackled with numerous contributions and different approaches. Among them, learning methods have been investigated in recent years due to the successful spreading of Artificial Intelligence and Machine Learning techniques. In particular, in reinforcement learning an agent learns by experience, i.e. through the interaction with the environment where it is placed, avoiding the need of a huge dataset for the training process. Service robotics is the main focus of the research at PIC4SeR (PoliTo Interdepartmental Centre for Service Robotics), where the idea of this thesis project is born as part of a broader project. Under the supervision of Professor M. Chiaberge (PoliTo), member of the group, the thesis embraces the vision of the centre, which is to develop high-tech solutions for peculiar fields such as precision agriculture, surveillance and security, in addition to assist people in their every-day life. As a matter of fact, an autonomous navigation system enables competitive advantages in a wide variety of the applications of interest. Deep Deterministic Policy Gradient (DDPG) is the specific DRL algorithm applied to train an agent in a simulated environment using ROS (Robot Operating System). Training simulations offer different types of scenarios presenting both static and moving obstacles. The main goal of the project is to provide a safe collision-free navigation in an unknown indoor environment. An Artificial Neural Network (ANN) is used to directly select suitable actions for the robot, expressed in terms of linear and angular velocity (ANN output). Input information is composed of robot pose and goal position, in addition to raw images provided by a depth camera. A great focus is also devoted to reduce the computational cost of the model in the training phase, as well as the energy consumption in a potential hardware implementation. For this reason, an efficient architecture of the Convolutional Neural Network (CNN) is studied, paying attention to both desired performances and costs. Firstly, a set of convolutional layers is needed to extract high-level features from depth images. Then fully-connected layers predict the action for the robot. Beside these aspects, also sensor data play a key role in a navigation system. From a research point of view, it is interesting to evaluate the performance of the algorithm when using depth images, compared to other popular implementations based on LIDAR sensor. On the one hand, depth camera offers a rich depth information. On the other hand a simple 2D LIDAR is able to cover a wider field of view. The navigation system has been tested in a virtual environment with obstacles. Despite the difficulty of the challenge and the amount of resources required for the development, the system can be considered a good starting point for future works. The implementation of the algorithm on a real robot will be a natural next step for the project.