Data Engineer
Core
Design, develop, and maintain data solutions for generation, collection, and processing, specifically focusing on migrating data from legacy on-prem systems to AWS.
Role type
Data Engineer
Builds
ETL pipelines and data migration solutions for internal systems and client data
Domain
Cloud Data Engineering, Data Migration
Deliverable
production ML models
Required skills
AWS Glue, PySpark, SQL, ETL, Unix, Iceberg, Astronomer, Data Warehousing concepts, Cloud computing services
Preferred skills
None stated
Technologies
AWS Step Functions, EventBridge, Lambda, CloudWatch
Responsibilities
Implement end-to-end ETL pipelines using AWS native services; Migrate high-volume datasets from on-prem to AWS; Monitor and troubleshoot pipelines for performance and cost; Support data quality validation and incremental loads; Collaborate with Solution Designers and DevOps engineers
Seniority
Mid-level, hands-on IC