HPC Energy Consumption Optimization

Datacenters play an increasingly crucial role in supporting a great variety of services, from cloud computing and data storage, to streaming services and artificial intelligence. Their growing high computing resources utilization, along with all the cooling system mechanisms to manage the generated heat, have led to significant environmental impacts and CO2 emissions. To address these challenges, this thesis proposes a predictive optimization mechanism aimed at reducing energy consumption and consequently environmental impacts. The research begins with the development of a Random Forest regressor capable of estimating the average power consumption of each submitted job, based on its resources demand specified by the user. A temperature model is then introduced, which uses the power consumption estimates to predict the temperature increments that would occur in the case of a specific job allocation on the available nodes. Several linear, non-linear, and neural networks models were trained, validated and tested, and after comparing their performance, the model which gave the best results was chosen. The core part is given by the optimization algorithm, which not only is able to handle job scheduling, but also manage node availability. In a few words, this optimizer can turn off nodes that are not required for current workloads, thus reducing unnecessary energy consumption, and then reactivate them when the job demand increases. Based on a Genetic Algorithm, the optimizer is designed to use the temperature model to predict future temperature behaviors, to make optimal decisions about job allocation. By considering the forecasted thermal state of the nodes, the algorithm determines the most suitable node for each job, aiming at distributing the workload avoiding excessive temperature increments. The implementation of this approach led to significant improvements in energy efficiency and thermal management. This work contributes to making datacenters more environmentally sustainable addressing key challenges in an era of growing digital demand.