Data Engineer Databricks (H/F)
Core
Design, develop, and optimize large-scale batch and streaming data pipelines on Databricks for clients in finance, industry, retail, mobility, and energy.
Role type
Senior Data Engineer (Databricks)
Builds
Production data pipelines and dataflows
Domain
Cloud Data Engineering / Big Data
Deliverable
production ML models | product features
Required skills
Advanced Databricks (Workspaces, Jobs/Workflows, Delta Lake, Unity Catalog), PySpark, Scala, Distributed processing, Spark optimization, CI/CD, Terraform
Preferred skills
Azure, AWS, GCP
Technologies
Databricks, PySpark, Scala, Delta Lake, Unity Catalog, Azure DevOps, GitHub Actions, Jenkins, Terraform
Responsibilities
Design and develop batch and streaming data pipelines on Databricks; Implement advanced transformations ensuring data quality, robustness, and performance; Industrialize dataflows using DataOps standards (CI/CD, automation, testing, versioning); Collaborate with Data Architects, Data Analysts, and Data Scientists; Act as a technical reference for development teams; Contribute to technology choices and prototypes.
Seniority
Senior, hands-on IC