Danial Soltanali Khalili
Dynamic Data Processing Pipelines: A Framework for Modular and Scalable Systems.
Rel. Alessandro Aliberti, Edoardo Patti. Politecnico di Torino, Corso di laurea magistrale in Digital Skills For Sustainable Societal Transitions, 2025
|
Preview |
PDF (Tesi_di_laurea)
- Tesi
Licenza: Creative Commons Attribution Non-commercial No Derivatives. Download (2MB) | Preview |
Abstract
This thesis introduces a framework for creating and managing dynamic data processing pipelines, focusing on the generation of self-sufficient executable Python code. The framework is designed to allow for the efficient description, compatibility, and execution of data processing tasks. It addresses the need for flexible pipeline creation in modern data environments, where existing tools often fall short in adaptability. Key aspects of the framework include: Dynamic pipeline creation based on user inputs, allowing for quick adjustments to workflows as data sources evolve. Ease of creating new processing blocks, which are self-contained executable Python scripts, enabling the system to expand its capabilities over time.
Handling of non-temporal data, particularly time series data, which is crucial for the GAIA platform for which the framework was initially developed
Relatori
Anno Accademico
Tipo di pubblicazione
Numero di pagine
Corso di laurea
Classe di laurea
Aziende collaboratrici
URI
![]() |
Modifica (riservato agli operatori) |
