Lead Data Engineer – Databricks - $180,000 - Gain "productionised" AI
Core
Lead Data Engineer building and supporting modern Databricks Lakehouse solutions on Azure to deliver scalable data pipelines, curated datasets, and trusted insights for enterprise BI, advanced analytics, and AI/ML workloads in a complex retail environment.
Role type
Lead Data Engineer (Azure/Databricks)
Builds
Scalable data pipelines, curated analytical datasets, Power BI semantic models, and production-ready data solutions for retail trading, operations, and customer use cases.
Domain
Retail, Cloud Data Engineering, Lakehouse Architecture
Required skills
Databricks (Unity Catalog, Delta Lake, Genie AI), PySpark, SQL, Python, Azure Databricks, Azure Data Factory, Azure Data Lake Storage, Power BI (DAX, semantic models), CI/CD, dimensional modelling, data governance
Preferred skills
Experience in fast-paced retail environments, stakeholder management, outcome-focused delivery
Technologies
Databricks, Unity Catalog, Delta Lake, Azure, Azure Data Factory, Azure Data Lake Storage, Power BI, Azure DevOps, PySpark
Responsibilities
Design and develop modern data engineering solutions on Azure; Build and optimise data pipelines and integrations; Develop and support Power BI datasets and semantic models; Provide BAU support for data platforms and pipelines; Implement CI/CD pipelines and apply cloud security best practices; Collaborate with stakeholders to translate requirements into technical solutions.
Seniority
Lead, hands-on IC with enterprise-scale experience
