CareerPlan

AWS Data Engineer

South AfricaFull-time2024-05-28 → 2026-11-16
Hybrid · South Africa

Core

Design, develop, and maintain scalable data infrastructure and pipelines on AWS, ensuring efficient data processing and storage.

Role type

Senior IC AWS Data Engineer

Builds

Scalable data pipelines, ETL processes, and data warehousing solutions on AWS

Domain

Cloud Data Engineering / AWS

Required skills

Python, PySpark, AWS Glue, SQL, AWS Athena, AWS Redshift, Data Warehousing, MDM (Master Data Management)

Preferred skills

AWS CloudFormation, Data Modeling, Data Governance, Git

Technologies

AWS (Glue, S3, Lambda, Redshift, Athena, Step Functions), Python, PySpark, SQL

Responsibilities

Design and maintain scalable data pipelines; Write efficient Python and PySpark code; Implement and manage ETL processes; Develop and optimize SQL queries; Utilize AWS Athena for querying large datasets; Work with AWS Redshift for data warehousing; Ensure data quality and integrity; Monitor and troubleshoot data pipeline issues.

Seniority

Senior, hands-on IC