Data Validation Engineer
Core
Implements technical controls for data parity, freshness, anomaly detection, pipeline observability, alerting, defect tracking, and release evidence to ensure data quality before customer release.
Role type
Data Validation Engineer
Builds
Cloud-based data and analytics platforms with governance and AI/ML capabilities for federal clients
Domain
Federal government data modernization and cloud analytics
Deliverable
production ML models
Required skills
Data quality automation, parity testing, freshness SLAs, anomaly detection, pipeline observability, data lineage, defect tracking, Databricks, Delta Lake, SQL, Python, Spark, Great Expectations, Monte Carlo, Unity Catalog, CI/CD automation
Preferred skills
Medallion architecture validation, MLOps, MLflow, Azure Machine Learning, statistical validation, drift detection, data governance controls
Technologies
Databricks, Delta Lake, Delta Live Tables, SQL, Python, Spark, Great Expectations, Soda, Monte Carlo, Unity Catalog, Collibra, Microsoft Purview, GitHub Actions, Azure DevOps, Terraform, MLflow, Azure Machine Learning
Responsibilities
Implement technical controls for data parity, freshness, anomaly detection, and pipeline observability; wire pipelines to monitoring and reporting mechanisms; collaborate with governance on rule libraries and quality thresholds; document work products and delivery evidence; support AI/ML workloads through training-data validation and model-input verification.
Seniority
Senior, hands-on IC