Study of the adaptability and efficiency of Reinforcement Learning based control for HVAC systems through EnergyPlus dynamic simulations.

The scope of this Master Thesis is to present a new application of an algorithm based on Reinforcement Learning (RL) for the control of HVAC systems to reduce the energy consumption. The aim of the analysis is to assess the adaptability of the RL algorithm to different operating conditions and to compare its performance against state of the art HVAC control strategies. The RL algorithm is allowed to choose the air temperature set-point of HVAC system within a band using as input, signals from sensors that are usually part of HVAC equipment and a weather forecast. Since RL algorithm learns online by interacting with an environment, the possibility offered by simulation tools is very attracting to study and develop such algorithms in the field of building energy management systems. In this paper, a study was conducted to test the performance of RL algorithm to the control of HVAC systems by means of dynamic building simulation conducted with the EnergyPlus (EP) tool. Starting from a real case study located in climate zone E of a Commercial Retail building, different models were developed within EP. Then this model is used to quantify energy savings obtained by RL control strategy. Then a building model based on Supermarket Reference Building (RB) proposed by DOE has been developed sizing the HVAC system with traditional design manuals and ASHRAE guidelines to assess energy savings introduced in comparison with state of the art for HVAC systems control. In particular, parameters such thermal insulation, occupancy, climate zone has been varied to assess the adaptability of the RL algorithm. RL is a class of machine learning algorithms born in the AI field that is showing promising results in optimal control applications. The HVAC state of the art controls comprehend rule-based or model-based controls. The former are difficult to optimally tune due to the intrinsic complexity of the coupled building and plant thermodynamics. The latter are strictly linked to the quality of the model of the specific building, which can be very expensive and complicated to be obtained at a sufficient level of fidelity in common working conditions. Both lack of performance when different conditions as weather, utilization of the building are met. RL control is able to guarantee the requirements pursuing near-optimal energy savings performance. Other attempts of using RL as control strategy for HVAC have already been experimented but extremely simplified models of buildings and HVAC plants have been used. The results has been obtained considering simulations two years long, performed using real meteorological data from various weather zones. For the real case study savings introduced by the algorithm are bigger because it can compensate some inaccuracies or changes in HVAC sizing or building destination where traditional controls lack in efficiency justifying the implementation of RL algorithm as retrofit application for existent buildings. RB model can simulate application in new buildings where the HVAC system is well sized and tuned for the building destination. In this case the RL algorithm is still able to achieve energy savings adjusting some parameters in the algorithm. The performances of the algorithm are constant when varying model parameters discussed before without any change in the algorithm while traditional controls are not able to guarantee energy savings or acceptable air temperature for the occupants.