Data Engineer - AWS/Databricks
Core
Design, build, and optimize scalable data pipelines and infrastructure on AWS and Databricks to support analytics and machine learning initiatives.
Role type
Data Engineer
Builds
Scalable data pipelines and ETL processes
Domain
Cloud data engineering (AWS, Databricks)
Deliverable
production ML models
Required skills
Python, Java, Scala, SQL, NoSQL, AWS (S3, Glue, EMR, Lambda, Redshift), Databricks, Docker, Kubernetes, Apache Airflow
Preferred skills
Data warehousing concepts, big data technologies, data governance and security best practices
Technologies
AWS, Databricks, S3, Glue, EMR, Lambda, Redshift, Python, Java, Scala, Docker, Kubernetes, Apache Airflow
Responsibilities
Design, develop, and maintain scalable data pipelines and ETL processes; Collaborate with data scientists and analysts to implement solutions; Optimize data storage and retrieval mechanisms; Implement data governance and security best practices; Troubleshoot and debug data pipeline issues; Stay abreast of emerging technologies and recommend innovative solutions
Seniority
Mid-level, hands-on IC
