Data Engineer - Big Data M/F
Core
Migrating and optimizing data processing solutions, specifically moving workloads from Cloudera to Databricks and building scalable real-time event processing architectures.
Role type
Senior Data Engineer (Big Data)
Builds
Production data pipelines and real-time event processing platforms
Domain
Big Data, Cloud Data Platforms, Real-time Streaming
Deliverable
production ML models | product features | infrastructure
Required skills
Apache Spark, Scala, Java, Confluent Kafka, Databricks, Cloudera, CI/CD, Git, Microservices architecture, Batch processing, Streaming processing
Preferred skills
Experience migrating from Cloudera to Databricks, Deep knowledge of Kafka production environments
Technologies
Databricks, Apache Spark, Scala, Confluent Kafka, Java, Cloudera, Git
Responsibilities
Migrate jobs and workflows from Cloudera to Databricks ensuring functionality and performance; Refactor and restructure data processing scripts; Build scalable architectures for large volume data via event streams; Develop data pipelines for both streaming and batch processing
Seniority
Mid-Senior, hands-on IC
