Senior Azure Data Engineer (Databricks)
Core
Design, build, and deploy scalable and secure data engineering pipelines using the Databricks platform on Azure for clients in a consulting environment.
Role type
Senior IC Azure Data Engineer (Databricks)
Builds
Production data pipelines, real-time event-driven solutions, and CI/CD infrastructure
Domain
Technology consulting, Cloud Data Engineering
Deliverable
production ML models | product features
Required skills
Databricks platform, PySpark, SQL, Delta Lake, Spark Structured Streaming, Unity Catalog, Azure-native services, Kafka, Azure Event Hubs, Azure DevOps, Jenkins, GitHub Actions, distributed data processing, Data Lakehouse architectures, schema design, GDPR compliance
Preferred skills
Scala, Java, Cloudera, Hadoop, Hive
Technologies
Databricks, Azure, Spark, Kafka, Unity Catalog, Delta Lake, Azure DevOps, Jenkins, GitHub Actions
Responsibilities
Design and develop robust data pipelines using Delta Lake, Spark Structured Streaming, Unity Catalog, and Azure-native services; Build real-time, event-driven solutions using technologies such as Kafka and Azure Event Hubs; Develop and maintain CI/CD pipelines using Azure DevOps, Jenkins, GitHub Actions; Collaborate with clients and multidisciplinary teams to translate business requirements into scalable, production-ready data solutions; Champion clean code, data lifecycle optimisation, engineering best practices, and the responsible use of approved AI tools and automation
Seniority
Senior, hands-on IC