Data Quality Engineer
Core
Design and implement data quality frameworks and automated checks across Azure and Databricks pipelines to ensure data reliability and consistency for business reporting.
Role type
Data Quality Engineer
Builds
Automated data quality checks, validation rules, and reconciliation processes for Bronze, Silver, and Gold data layers.
Domain
Cloud data platforms (Azure, Databricks) and data engineering
Deliverable
production ML models | product features
Required skills
Azure, Databricks, SQL, data quality frameworks, ETL/ELT validation, layered data architecture, pipeline monitoring, automated validation, data reconciliation
Preferred skills
Identity resolution, UAT support, incident triaging, documentation of QA processes
Technologies
Azure, Databricks
Responsibilities
Design data quality frameworks with validation rules and alerting standards; Build automated checks for row counts, nulls, referential integrity, and schema validation; Own reconciliation between source systems and Databricks layers; Validate identity resolution outputs in the Silver layer; Perform end-to-end pipeline testing from ingestion to Gold layer; Partner with Data Engineers to define acceptance criteria for pipeline deliverables; Support UAT with client business stakeholders; Document QA processes and findings for client handoff; Monitor pipeline health and triage data quality incidents.
Seniority
Mid-level, hands-on IC