Data Validation Engineer
Core
Implements technical controls for data parity, freshness, anomaly detection, pipeline observability, alerting, and defect tracking to ensure data quality before release.
Role type
Data Validation Engineer
Builds
Cloud-based data and analytics ecosystem with storage, analytics, governance, and AI/ML capabilities
Domain
Federal government data modernization and cloud analytics
Deliverable
production ML models
Required skills
Data quality automation, parity testing, freshness SLAs, anomaly detection, pipeline observability, data lineage, defect tracking, SQL, Python, Spark, Databricks, Delta Lake, Great Expectations, Monte Carlo, Unity Catalog, Azure Data Factory, MLOps
Preferred skills
Automated data quality frameworks, reconciliation processes, data certification workflows, drift detection, medallion architecture validation, operational dashboards, DataOps practices, AI/ML workload validation, model-input/output verification
Technologies
Databricks, Delta Lake, Delta Live Tables, SQL, Python, Spark, Great Expectations, Soda, Monte Carlo, Deequ, Unity Catalog, Collibra EDC, Microsoft Purview, Azure Data Factory, MLflow, Azure Machine Learning
Responsibilities
Implement technical controls for data parity, freshness, anomaly detection, pipeline observability, alerting, and defect tracking. Wire pipelines to monitoring and reporting mechanisms. Collaborate with governance on rule libraries and quality thresholds. Document work products, decisions, risks, and delivery evidence.
Seniority
Senior, hands-on IC