Model Predictive Control and Reinforcement Learning for Quadrotor Agile Flight Control

The work presented in this master thesis project is related to the control aspects of fast and agile drone trajectory tracking. Aerodynamic forces make quadrotors trajectory tracking at high-speed extremely challenging. At high speeds these complex effects have a major impact in performance loss, measured in terms of large position tracking errors. Model Predictive Control (MPC) together with Reinforcement Learning (RL) is used to tackle the problem. We propose to use RL to offline tune the MPC formulation using the data obtained from the system. MPC is an optimal control method with a well-established theory that exploits a dynamic model of the platform and provides constraint satisfaction. RL methods allow solving control problems with minimum prior knowledge about the task. RL automatically trains the decision-making process via trial and error and maximize the performance through a given reward function. In our approach, RL is used for adjusting the MPC parameters, through a Q-learning technique by exploiting MPC as a function approximator. Indeed, unlike Deep Neural Networks (DNN), MPC as a function approximator for RL, can explicitly achieve constraints satisfaction, stability, and safety. Therefore, the goal is to combine the advantages of both methods: the ability of MPC to safely control a physical robot through well-established knowledge and the power of RL to learn complex policies using experienced data. The resulting control framework can handle large-scale inputs, reduce human intervention in design and tuning, and eventually achieve optimal control performance. The method is verified through precise and extensive simulation environment. This work is the result of a seven months period at the Norwegian University of Science and Technology (NTNU), Trondheim, Norway under the group of Professor Sebastein Gros, supervised by Postdoctoral fellow Dirk Reinhardt.