Data Engineer - Spark AWS
Core
Design and evolve large-scale data processing pipelines and develop robust distributed treatments primarily in Scala.
Role type
Senior Data Engineer (Spark/AWS)
Builds
Large-scale data processing pipelines and distributed treatments
Domain
Data Engineering / Cloud Infrastructure
Deliverable
production ML models | product features
Required skills
Apache Spark, Scala, AWS, SQL, Airflow, Docker, CI/CD, observability, code reviews, performance optimization, debugging
Preferred skills
Architecture decision making, software quality, industrialization of pipelines
Technologies
Apache Spark, Scala, AWS, Airflow, Docker, SQL
Responsibilities
Design and evolve large-scale data processing pipelines, Develop robust distributed treatments in Scala, Participate in architecture choices and technical decisions, Optimize performance and costs of treatments, Ensure data quality and reliability in production, Investigate and resolve complex problems (performance, debugging), Collaborate with Data, Product, and Backend teams
Seniority
Senior, hands-on IC
