INVERSE REINFORCEMENT LEARNING FOR AUTONOMOUS DRIVING

This project has been developed in collaboration with my team co-worker Francesco Allegra, a mechatronic engineering student, at Addfor S.p.A., with the guidelines of our academic supervisor Stefano Alberto Malan. The writing of the thesis has been divided as follow: I wrote chapters 2-5-6 while Francesco Allegra wrote chapters 1-3-5 and the results achieved so far have been summarized in “Conclusion”. In this thesis It has been addressed the problem of using Inverse Reinforcement learning algorithms in order to realize autonomous driving’s applications.Among all the possible goals, “autonomous driving” is one of the most challenging, since it is a task performed by a huge amount of people everyday, and that is becoming more and more complex as the number of vehicles grows up during the years. The complexity of driving lies in the fact that a driving scenario is strongly unpredictable and unstructured, either in highways and especially in a city. Furthermore, it has been shown that most of car accident causes can mostly be attributed to human-driver’s misjudgments or distractions. These are some of the reasons which led to the introduction of ADAS in modern urban cars, that assists the driver in some of the fundamental tasks that can be executed while driving (regulate the acceleration, parking, maintain the lane), but according to the SAE we are still far from the highest level of automation (Level 5) that indicates the ability of the car of driving by itself without any human input. At the moment, ADAS are implemented through classical control techniques, e.g. P.I.D, P.I or P.D., or using some advanced control theories like Model predictive control.Simultaneously, Artificial Intelligence (AI) has acquired a lot of importance, spreading in a wide range of engineering sectors since it provides solutions in a very efficient and cleaver manner. Reinforcement Learning, the third branch of ML, can effectively be involved as a decision-making problem solver, and it is just here that the control problem in autonomous driving meets machine learning.The focal point of RL is the Reward, a feedback signal able to show how good or bad are the taken action, so RL is the right choice only if we are dealing with tasks that can be accomplished in a small environment requiring discrete state and action spaces, instead if we focus our attention on complex scenarios with continuous spaces, it leads to wrong result, since we can’t guarantee the correctness of the chosen reward; those limits opened the door to a new theory, introduced by Andrew Ng and Russel in 2000s as inverse reinforcement learning, which aim is to find the reward in the cases cited before. The objective of this thesis is to explain how to use the projection-based method, an inverse reinforcement learning algorithm, to realize an adaptive cruise control and a lane keeping control, given a set of data provided by Addfor S.p.A., which describes the driving style of two different expert human drivers that our agent should emulate. The entire code has been written in MATLAB while the scheme containing the agent block, the vehicle dynamic block and the sensors blocks, has been realized in Simulink. Moreover, driving scenario Toolbox has been used to build a driving scenario ad hoc for the problem. In the Conclusion It has been discussed the advantages and disadvantages of applying this procedure by comparing the results of the simulations with the ones obtained from already known techniques.