Senior Databricks Migration Engineer
Core
Senior data engineer leading the migration of legacy SQL Server stored procedures and Azure Data Factory pipelines to a Databricks Lakehouse architecture.
Role type
Senior IC data engineering lead (cloud migration & Lakehouse)
Builds
Scalable Bronze, Silver, and Gold Lakehouse frameworks using Delta Live Tables and Databricks Workflows
Domain
Cloud data engineering (Azure, Databricks, Delta Lake)
Required skills
PySpark, Delta Lake, Unity Catalog, Azure Data Factory, SQL, dimensional modeling, data pipeline development, query performance optimization, cloud architecture
Preferred skills
Power BI optimization, serverless SQL compute, cost optimization strategies, security governance implementation
Technologies
Databricks, Apache Spark, Delta Lake, Azure, Power BI, Unity Catalog, Microsoft Entra ID
Responsibilities
Lead migration of legacy SQL Server stored procedures and Azure Data Factory pipelines to Databricks Lakehouse; Design reusable ETL/ELT frameworks using PySpark and Delta Live Tables; Architect and optimize Gold Layer dimensional models for Power BI; Implement advanced Delta Lake optimization techniques; Define cluster sizing and auto-scaling standards; Establish data security and governance using Unity Catalog; Lead pair-programming sessions and technical workshops for team transition.
Seniority
Senior, hands-on IC with technical leadership (via careerplan.io/jobs/e524fa7c-6f39-4ff3-9764-21cd059c9bb7-senior-databricks-migration-engineer-at-jobgether)