Data Engineer - AWS, Pyspark, Databricks
Core
Design and build scalable AWS and PySpark data pipelines, drive data quality and reconciliation, and support a major cloud lakehouse migration.
Role type
Senior Data Engineer (Cloud Data Platform)
Builds
Scalable data pipelines and modern lakehouse data platforms
Domain
Cloud Data Engineering / Big Data
Deliverable
production ML models
Required skills
PySpark, AWS (Glue, Athena, S3, Lambda, Step Functions), Python, SQL, Data Modeling, Data Quality, Reconciliation, CI/CD, Git, Infrastructure as Code
Preferred skills
Databricks, Delta Lake
Responsibilities
Design and build scalable AWS and PySpark data pipelines; Drive data quality, reconciliation, and platform reliability; Solve complex data challenges and optimise large-scale transformations; Support a major cloud lakehouse and Databricks migration initiative; Partner with business and technology stakeholders to deliver trusted data outcomes