Data Engineer
Core
Design, develop, and maintain data solutions for data generation, collection, and processing, focusing on migrating and deploying data across systems.
Role type
Senior Data Engineer (ETL & Data Migration)
Builds
End-to-end ETL pipelines and data migration solutions
Domain
Cloud Data Engineering (AWS)
Deliverable
production ML models | product features
Required skills
PySpark, SQL, ETL, Unix, Data Warehousing concepts, Cloud computing services, Data pipeline architecture
Preferred skills
AWS Glue, Iceberg, Astronomer, Step Functions, EventBridge, Lambda, CloudWatch
Technologies
AWS Glue, PySpark, SQL, Unix, Iceberg, Astronomer, Step Functions, EventBridge, Lambda, CloudWatch
Responsibilities
Implement end-to-end ETL pipelines using AWS native services; Handle high-volume datasets with error handling and retry mechanisms; Collaborate with Solution Designer and DevOps to test and deploy pipelines; Monitor and troubleshoot pipelines for performance and cost optimization; Support data quality validation and incremental loads; Engage with multiple teams to provide solutions.
Seniority
Senior, hands-on IC
