Marco Angius
Log Mining for Failure Analysis on Spark.
Rel. Paolo Garza. Politecnico di Torino, Corso di laurea magistrale in Ict For Smart Societies (Ict Per La Società Del Futuro), 2019
Abstract
In the past few years Amadeus has decided to rely on Big data platforms to develop advanced analytics modules and help airlines to improve their monitoring on performance insights. Big data platforms have matured since the introduction of Hadoop more than 10 years ago. Today Apache Spark represents the next-generation of data processing frameworks, providing ad- vanced in-memory capabilities and a directed acyclic graph (DAG) engine. Spark was developed to address the limitation of MapReduce, being 10–100 times faster in most of the data processing workloads. Nevertheless, Spark presents cases of application failures which are very difficult to interpret and, therefore, correct.
This work presents a deep predictive data analysis per- formed on Spark application logs in order to discover failure patterns and speed up the issue resolutions
Relatori
Anno Accademico
Tipo di pubblicazione
Numero di pagine
Informazioni aggiuntive
Corso di laurea
Classe di laurea
Ente in cotutela
URI
![]() |
Modifica (riservato agli operatori) |
