Federico Baldi
DataOps Pipelines and LLM-Driven Self-Healing: A Reference Architecture for Resilient Data Operations.
Rel. Paolo Garza. Politecnico di Torino, Corso di laurea magistrale in Data Science And Engineering, 2026
|
Preview |
PDF (Tesi_di_laurea)
- Tesi
Licenza: Creative Commons Attribution Non-commercial No Derivatives. Download (7MB) | Preview |
Abstract
The exponential growth in enterprise data volume and complexity has highlighted the limits of monolithic data architectures, leading to the adoption of decentralized paradigms such as Data Mesh and DataOps engineering methodologies. Although these approaches aim to resolve issues in data team workflows, their implementation in production environments remains inconsistently handled, particularly regarding data quality assurance, pipeline observability and controlled error management across the ETL/ELT lifecycle, leading to extended Mean Time to Recovery (MTTR). The first stage of this thesis is the design, implementation and validation of a DataOps-oriented data pipeline, built on the European Soccer Database as an experimental basis.
Structured according to Domain-Driven Design principles within a Medallion Architecture, and fully implemented as Infrastructure-as-Code (IaC) via Databricks Asset Bundles (DABs) and Data Build Tool (dbt), the pipeline integrates automated data quality testing against core KPIs – completeness, validity, freshness, and consistency – alongside systematic observability of job performance, execution failures and anomaly detection
Relatori
Anno Accademico
Tipo di pubblicazione
Numero di pagine
Corso di laurea
Classe di laurea
Aziende collaboratrici
URI
![]() |
Modifica (riservato agli operatori) |
