Reinforcement Learning for the scheduling of EV charging systems in a Smart Grid context

In the last decades global warming has aroused increasing attention as new temperature records are being registered worldwide and new violent and unpredictable meteorological events occur. Italy has just registered the second hottest summer in the last 200 years and different countries have registered the hottest temperatures ever. Climatologists agree that these temperatures are five times more probable due to climate change and they are caused by the increasing amount of CO2 and other greenhouse effect gases released on the atmosphere due to human activity. Organizations at national and international level are constantly studying and monitoring the production of greenhouse effect gases and for both Europe and the USA it emerges that in the last years the domestic transportation sector is responsible for about 22-27% of emissions, and that the energy production sector is responsible for about 25% of emissions. If combined, different enabling technologies that has been developed in the last decades can give an enormous contribution to the reduction of emissions produced by domestic transportation. These technologies include: the improvement of the quality and capacity of electric batteries and their use in electric vehicles, the increased efficiency and cost reduction of photo-voltaic panels for local energy production, the improvement of ICT and electrical infrastructure for the operation of Smart Grids, and the development of new computational algorithms able to control complex systems under uncertainty in a reasonable time. This work aims at studying different algorithms for scheduling the recharge of batteries in a Battery Swapping Station (BSS) which are stations in which electric vehicles drop their empty battery, and pick a fully charged one to use. These algorithms take into account that the station may be provided a small photo-voltaic plant for local energy production, and that it may be embedded in a Smart Grid in which the cost of electricity varies during the day. The aim of the control algorithm is to reduce as much as possible both the electricity consumption from the grid, and the number of cars that cannot be served due to the lack of charged batteries in the station considering that there is a natural trade-off between these two performance indicators.