Development of a Reinforcement Learning based Energy Management System (EMS) for a plug-in Hybrid Electric Vehicle

As the world strives in the pursue of carbon neutrality, Hybrid Electric Vehicles (HEVs) raised as one of the most promising solutions to reduce greenhouse gas emissions. In this context, an effective and reliable Energy Management System (EMS) becomes essential to fully exploit the hybrid potential since the presence of two energy sources (i.e. the fuel tank and the chemical battery) introduces an additional degree of freedom which has to be properly handled. To address this control problem, in the wide portfolio of techniques available, Artificial Intelligence (AI) represents nowadays the cutting-edge solution. Reinforcement Learning (RL) seems to be one of the most promising thanks to its peculiar structure, where an agent interacts directly with an environment, making decisions and receiving feedback in the form of rewards. In this study, a new Soft Actor-Critic (SAC) agent, which exploits a stochastic policy, was implemented and trained on a digital twin of a state-of-the-art diesel Plug-in Hybrid Electric Vehicle (PHEV) available on the European market. The training procedure was performed over the Worldwide harmonized Light-duty vehicles Test Cycle (WLTC) in order to consider driving conditions representative of the real vehicle usage. Finally, a sensitivity analysis was performed on different rewards by testing multiple penalty functions aimed at enhancing the fuel economy while guaranteeing the battery charge sustainability. The simulation platform was built in the Python environment thanks to its flexibility and robust support for implementing and assessing the SAC agent performances compared to the benchmark set by Dynamic Programming (DP). The SAC was able to achieve energy consumption levels close to the DP results, with differences lower than 10% but with the great additional benefit of being suitable for online applications.