Lead Data Engineer - Data Migration & Lakehouse
Core
Lead large-scale data migrations from legacy systems (mainframes, SQL, ERP) into modern cloud/lakehouse architectures, designing robust pipelines and establishing data quality frameworks.
Role type
Senior Lead Data Engineer (Data Migration & Lakehouse)
Builds
Modern cloud/lakehouse data architectures and automated validation pipelines
Domain
Data Engineering / Cloud Data Platforms
Deliverable
production ML models
Required skills
Large-scale legacy-to-cloud migration leadership, Databricks ecosystem mastery (Spark, Delta Lake, Unity Catalog, Lakeflow, Asset Bundles), Data modeling & schema mapping across heterogeneous systems, Discovery & gap analysis, Data quality framework design, Stakeholder risk communication
Preferred skills
Legacy system archaeology (COBOL, EDI), Agentic AI exposure, Metadata management beyond Unity Catalog, LLM-based mapping/matching tooling
Technologies
Databricks, Spark, Delta Lake, Unity Catalog, Lakeflow, Asset Bundles, Great Expectations, dbt
Responsibilities
Lead end-to-end migration projects from legacy to cloud/lakehouse, Design and implement CI/CD-style data pipelines, Own data modeling and reconciliation across heterogeneous systems, Conduct discovery and assessment of legacy systems, Establish data quality and automated validation standards, Surface risks and technical gaps to stakeholders
Seniority
Senior, hands-on IC with leadership responsibilities