Imitation Learning for Dexterous Robotic Manipulation

This thesis explores imitaion learning approaches for dexterous robotic manipulation, addressing the limitations of reinforcement learning and traditional behavioral cloning when applied to complex manipulation tasks. While reinforcement learning struggles because of the definition of an effective reward function, behavior cloning suffers from compounding errors and poor generalization when out-of-distribution states are reached during deployment. This works aims to answer the question on whether the use of a simulator can help improve on existing state of the art baselines, and therefore proposes a new algorithm that makes use of both expert demonstrations and samples from the environment. The key components of this project were the creation of a model of the robotic arm with its hand, teleoperation-based data collection using a virtual reality interface, and sim to real mapping for hand control, as well as the study and development of different algorithms. The core contribution is the BC+PPO algorithm, which combines maximum likelihood behavior cloning estimation within a proximal policy optimization training loop, thus constructing a hybrid reward function. The proposed method aims to address the difficulty in the use of reinforcement learning for manipulation tasks, by requiring only basic and easy to define reward functions, guiding the overall behavior of the robot by sampling from given expert demonstrations. Experimental evaluation on manipulation tasks including lift and pick-and-place demonstrate competitive performance on high-dimensional state space systems.