AWS Glue Data Engineer
Core
Building scalable ingestion pipelines to migrate source systems into a Lakehouse ingestion zone, focusing on Bronze layer data loading and performance optimization.
Role type
Senior AWS Glue Data Engineer
Builds
Scalable ETL pipelines and data ingestion jobs for a Lakehouse architecture
Domain
Financial services / Data Engineering
Deliverable
production ML models
Required skills
AWS Glue, PySpark, ETL pipeline development, Lakehouse architecture, Medallion design principles, CDC, delta loads, historical data ingestion strategies, Git, Terraform
Preferred skills
AWS services (S3, Athena, Lambda), data quality frameworks (Soda Core), CI/CD automation, agile methodology
Technologies
AWS Glue, PySpark, S3, Athena, Lambda, Git, Terraform, Soda Core, Jira
Responsibilities
Building and implementing AWS Glue jobs for Bronze layer ingestion, optimizing Glue job performance (DPU allocation, parallelization), migrating source tables to Bronze layer, ensuring jobs comply with data contracts and are monitored, collaborating with Data Architect and Data Assurance Lead
Seniority
Senior, hands-on IC
