Automation of ETL Pipelines in DataStage
Alex Umberto Benedetti
Automation of ETL Pipelines in DataStage.
Rel. Guido Albertengo. Politecnico di Torino, Corso di laurea magistrale in Ict For Smart Societies (Ict Per La Società Del Futuro), 2025
|
Preview |
PDF (Tesi_di_laurea)
- Tesi
Licenza: Creative Commons Attribution Non-commercial No Derivatives. Download (3MB) | Preview |
Abstract
In today’s business ecosystem for enterprise data management, the automation process of ETL (Extract Transform Load) pipelines has become one of the primary objectives. This master’s thesis explores the use of latest Artificial Intelligence techniques to simplify data integration and transformation, with the aim of minimizing the human effort in designing and managing workflows to process data. Conducted in partnership with Mediamente Consulting Srl, this research aims to design and implement a system that examines and utilizes advanced technologies to effectively manage user requests in an ETL data flow context. The central process involves the automation of DataStage components using XML templates to dynamically create and configure jobs based on the interpreted user requests.
Through the development of custom scripts, the system automates the deployment and configuration of DataStage jobs, transforming the ETL setup from a manual, error-prone process into a more efficient and reliable automated procedure
Tipo di pubblicazione
URI
![]() |
Modifica (riservato agli operatori) |
