Data Engineer Databricks Senior - H/F - CDI
Core
Senior Data Engineer managing a team to build, optimize, and maintain big data pipelines on Databricks for data acquisition, preparation, modeling, and storage.
Role type
Senior IC Data Engineer (Team Lead)
Builds
Data pipelines and data platforms on Databricks
Domain
Big Data, Cloud Data Engineering
Deliverable
production ML models
Required skills
Spark, Databricks, Scala, Python, SQL, Data pipeline development, Cloud platforms (AWS), DevOps tools (Jenkins, GitLab, Maven), HDFS/S3 storage
Preferred skills
Real-time streaming (Kafka + Spark Streaming), DataOps, DataVault, DataMesh concepts
Technologies
Databricks, Spark, AWS, Kafka, Jenkins, GitLab, Maven, HDFS, S3
Responsibilities
Manage Big Data and Cloud Engineers, provide technical coaching and code reviews, define solution architecture, develop and optimize data pipelines, perform benchmarking and client consulting, share knowledge and conduct internal training
Seniority
Senior, hands-on IC with team management
