Senior Data Engineer
Core
Build, extend, and maintain an enterprise Lakehouse platform on Databricks to support reporting, analytics, and AI initiatives while migrating legacy SQL Server environments.
Role type
Senior IC data engineer with line management responsibilities
Builds
Governed, interoperable data estate on Databricks (Lakehouse architecture)
Domain
Cloud data engineering, Azure ecosystem, data warehousing
Required skills
Python (PySpark), SQL, Databricks, Azure data storage, ETL/orchestration, data governance, CI/CD, dimensional modelling
Preferred skills
AI-assisted development tools, legacy SQL Server migration
Technologies
Databricks, PySpark, Spark SQL, Delta Lake, Azure SQL, Data Factory, Azure DevOps, Git, Unity Catalog
Responsibilities
Provide technical leadership and direction; Design and optimize data pipelines using PySpark and Delta Lake; Implement data validation and automated testing; Lead migration of legacy SQL Server processes; Manage and mentor junior/mid-level engineers; Drive adoption of modern engineering practices and automation
Seniority
Senior, hands-on IC with line management