polito.it
Politecnico di Torino (logo)

An end-to-end Data Engineering Architecture in the energy domain

Michele Apicella

An end-to-end Data Engineering Architecture in the energy domain.

Rel. Paolo Garza. Politecnico di Torino, Corso di laurea magistrale in Data Science And Engineering, 2022

[img]
Preview
PDF (Tesi_di_laurea) - Tesi
Licenza: Creative Commons Attribution Non-commercial No Derivatives.

Download (3MB) | Preview
Abstract:

The idea behind this thesis work is to create an end-to-end Data Engineering use case for Bamboo Energy, a tech start-up operating in the energy sector. The development of this work has been made possible firstly doing an internship and then being included in the product team. The overall research and implementation process can be conceptually divided into four different phases. The first step consisted in the study of the high level processes of Bamboo Energy, its main products and its features, its placement in the industry sector and the figures involved in the energy paradigm. After, the focus has been moved into topics related to Data Engineering, covering ETL and aggregation pipelines, Orchestration and Alerting processes. The goal was to make a snapshot of Bamboo Energy situation related to this fields, in order to identify strengths and weaknesses, to have a base from which begin a research and exploration work. The third phase can be identified in the pure exploration of possible new paths, comparing and studying different technologies. Finally, among different solutions, some have been chosen to be implemented, utilizing, as methodology, product and budget constraints, platform and cloud environment compatibility. The outcome is a very practical end-to-end use case covering aspects from data sources ingestion to transformation and aggregation pipelines, from processes Orchestration to Alerting policies.

Relators: Paolo Garza
Academic year: 2022/23
Publication type: Electronic
Number of Pages: 86
Subjects:
Corso di laurea: Corso di laurea magistrale in Data Science And Engineering
Classe di laurea: New organization > Master science > LM-32 - COMPUTER SYSTEMS ENGINEERING
Aziende collaboratrici: Bamboo Energy Platform S.L.
URI: http://webthesis.biblio.polito.it/id/eprint/24613
Modify record (reserved for operators) Modify record (reserved for operators)