Data Engineer
Core
Design, develop, and maintain data solutions for data generation, collection, and processing, focusing on migrating and deploying data across systems.
Role type
Data Engineer (ETL & Data Migration)
Builds
ETL pipelines and data migration solutions
Domain
Cloud Data Engineering (AWS)
Deliverable
production ML models | product features
Required skills
AWS Glue, PySpark, SQL, ETL, Unix, Iceberg, Astronomer, Data Warehousing concepts
Preferred skills
Data pipeline architecture design, Cloud computing services, Data storage solutions
Technologies
AWS Step Functions, AWS EventBridge, AWS Lambda, CloudWatch
Responsibilities
Implement end-to-end ETL pipelines using AWS native services; Handle high-volume datasets with error handling and retry mechanisms; Monitor and troubleshoot pipelines for performance and cost optimization; Support data quality validation and incremental loads; Collaborate with Solution Designers and DevOps engineers for testing and deployment.
Seniority
Mid-level (3+ years experience)