The recent rise of Advanced Driver Assistance Systems (ADAS) and connectivity in the automotive field applied to Connected and Autonomous Vehicles (CAVs) has led the research efforts to investigate new control strategies for improving Mobility solutions. An alternative approach that gained recent interest and that can be applied to complex control problems is Reinforcement Learning (RL). RL is a branch of Machine Learning that consists of training an Agent to behave in a desired manner, and that has been recently proven to reach comparable or enhanced performance concerning more common optimal control strategies such as Model Predictive Control, especially in terms of computational costs and in presence of high dimensional and uncertain environments.