Sobre el puesto
OverviewIn this role you will design and implement an AI-powered data validation platform for ETL pipelines on AWS, ensuring data quality and governance.
You will work with cross-functional teams to automate validation, detect anomalies, and explain issues before they affect downstream systems.
The role combines data engineering, ML-driven validation, and generative AI to scale quality checks across workloads.
You will shape the validation roadmap and contribute to data quality strategy while delivering actionable insights.
ResponsabilidadesAnalyze existing ETL validation processes and identify automation opportunities
Develop AI-driven validation frameworks for data pipelines
Implement anomaly detection for continuous data quality monitoring
Compare source data against ETL outputs to identify and track discrepancies
Automate record count, completeness, and consistency checks across pipelines
Generate AI-assisted validation reports that explain detected issues in clear, actionable terms
Integrate AI validation capabilities into AWS data platforms and workflows (Glue, Lambda, Redshift, Athena, S3)
Collaborate with data engineering, QA, and business teams to define data quality rules, thresholds, and SLAs
Roadmap for expanding AI-driven validation across additional workloads
Requisitos principalesBachelor's degree in Computer Science, Data Engineering, Software Engineering, or related field (or equivalent experience)
5+ years of experience in data engineering, ETL development, or data quality roles
Strong programming skills in Python and advanced SQL
Hands-on experience with Amazon Bedrock and building generative AI solutions (prompt design, LLM-based analysis and reporting)
Hands-on experience with AWS Lambda, AWS Glue, Amazon Redshift, Amazon Athena, and Amazon S3
Practical experience with machine learning and anomaly detection techniques applied to data (statistical, time-series, or ML-based)
Solid understanding of data quality and data governance principles (completeness, accuracy, consistency, lineage)
Experience designing source-to-target reconciliation and ETL testing strategies
Solid understanding of Agile, SDLC, and modern engineering best practices
collaboration with cross-functional teams
clear communication of complex concepts
problem solving and initiative
Python
SQL
Amazon Bedrock