Data QA Automation Engineer
Core
Design, build, and maintain automated testing frameworks, data quality gates, and regression suites for EDW delivery teams to enforce quality in CI/CD pipelines.
Role type
Senior IC data QA automation engineer
Builds
Automated regression suites, data quality gates, and test data management utilities for data pipelines
Domain
Financial services data engineering and quality assurance
Required skills
Python, SQL, data testing frameworks (pytest, Great Expectations, dbt tests, Soda), CI/CD integration, ETL/ELT patterns, data warehouse concepts, test automation, code review, technical documentation
Preferred skills
Databricks (PySpark, Delta Lake, Unity Catalog), performance/load testing, data observability, anomaly detection tooling
Technologies
Azure DevOps, Git, Python, SQL, SSIS, dbt, Fivetran, Databricks, Delta Lake, Unity Catalog, pytest, Great Expectations, Claude, Copilot
Responsibilities
Design and implement automated data quality CI/CD gates; build reusable regression suites and source-to-target reconciliation; integrate automated tests into pipelines; generate synthetic and masked test data; validate AI-assisted test code; mentor engineers on automation standards; measure automation ROI
Seniority
Senior, hands-on IC