Politecnico di Torino (logo)

Deep Reinforcement Learning for Horizontal Autoscaling: a Proof of Concept

Dario Salza

Deep Reinforcement Learning for Horizontal Autoscaling: a Proof of Concept.

Rel. Paolo Garza. Politecnico di Torino, Corso di laurea magistrale in Ingegneria Informatica (Computer Engineering), 2020


Horizontal Autoscaling has become a fundamental feature of modern container orchestration systems, allowing the hosted services to scale dynamically following the demand, servicing it in the respect of the Service Level Agreements, while avoiding wasting resources with fixed or too generous allocations. In the industry, threshold-based autoscalers are still the standard, given their simplicity, but they do not always guarantee optimal performances nor real ease of configuration: as they operate reactively, they can be unstable in high variance environments, and alternatively require very conservative thresholds or risk arriving late in case of rapid demand growths. Given the complex nature of the context, Deep Reinforcement Learning techniques can be used both to automatize the policy tuning, both to discover new policies having more expressivity and better performances than those possible with threshold-based systems. After the definition of an exemplificatory target environment and the formalization of the autoscaling problem as a Markov Decision Process, several possible approaches are hereby evaluated, comparing their performance with tuned threshold-based systems. The developed final methodologies outperform all the tested threshold-based configurations in the considered environments, presenting better SLA violations/used resources ratios and decreasing the severity of SLA violations. As the training procedure shows however considerable caveats over its real-world applicability, the discussed approach is expanded to include Meta Reinforcement Learning principles, showing promising preliminary results that could indicate the effective possibility of, upon further developments, industry-level applications.

Relators: Paolo Garza
Academic year: 2019/20
Publication type: Electronic
Number of Pages: 70
Additional Information: Tesi secretata. Fulltext non presente
Corso di laurea: Corso di laurea magistrale in Ingegneria Informatica (Computer Engineering)
Classe di laurea: New organization > Master science > LM-32 - COMPUTER SYSTEMS ENGINEERING
Ente in cotutela: TELECOM ParisTech - EURECOM (FRANCIA)
Aziende collaboratrici: Nokia Bell Labs France
URI: http://webthesis.biblio.polito.it/id/eprint/14381
Modify record (reserved for operators) Modify record (reserved for operators)