Data Engineer Databricks
Core
Design, develop, and optimize robust data pipelines on AWS using Databricks and Apache Spark.
Role type
Data Engineer (Databricks)
Builds
Data pipelines and data sets for data science and analytics teams
Domain
Cloud data engineering (AWS)
Deliverable
production ML models | product features
Required skills
Databricks, Apache Spark, Python, AWS services (S3, Redshift, EMR), CI/CD automation, performance optimization, scalability design
Preferred skills
Incident management, monitoring setup, logging implementation
Technologies
Databricks, Apache Spark, Python, AWS S3, AWS Redshift, AWS EMR
Responsibilities
Design, develop, and maintain robust data pipelines; Integrate and orchestrate data flows on AWS; Optimize Spark job performance and ensure solution scalability; Collaborate with data science and analytics teams to provide reliable datasets; Implement monitoring, logging, and incident management processes
Seniority
Mid-level (3-5 years experience)
