Senior Databricks Data Engineer
Core
Design and implement modern data platforms, AI workflows, and Generative AI solutions using Databricks and Azure technologies.
Role type
Senior IC data engineer (AI/ML focus)
Builds
Production-grade ETL/ELT pipelines, Lakehouse architectures, RAG pipelines, multi-agent AI workflows, and MLOps infrastructure.
Domain
Enterprise data engineering and Generative AI on Azure cloud
Deliverable
production ML models
Required skills
Python, SQL, PySpark, Delta Lake, Unity Catalog, Databricks Workflows, MLflow, Azure Service Bus, Azure Functions, Azure Data Factory, Git, CI/CD
Preferred skills
PyTorch, TensorFlow, Event Hub, Logic Apps, enterprise-scale AI platform design, mentoring
Technologies
Databricks, PySpark, Delta Lake, Unity Catalog, MLflow, Python, SQL, Apache Spark, Azure Databricks, Azure Service Bus, Azure Functions, Azure Data Factory, Azure Data Lake Storage Gen2, Azure Key Vault, Azure DevOps, Git, PyTorch, TensorFlow
Responsibilities
Design scalable ETL/ELT and AI pipelines; Develop and maintain Lakehouse architectures; Build and optimise Delta Live Tables and job orchestration; Develop and deploy production-grade Generative AI and agent-based solutions; Create and maintain RAG pipelines and AI agent frameworks; Support MLOps activities including model training and inference pipelines.
Seniority
Senior, hands-on IC
