Data Engineer (Senior) 1053
Core
Design, build, and maintain scalable data pipelines and ETL workflows on cloud platforms.
Role type
Senior Data Engineer
Builds
Scalable data pipelines, ETL workflows, data lakes, and data warehouses
Domain
Cloud data engineering (AWS/OCI)
Deliverable
production ML models | product features | infrastructure
Required skills
Python, PySpark, AWS services (Athena, Glue, Lambda, S3, ECS), NoSQL (DynamoDB), relational databases (Oracle/PostgreSQL), OCI services, data formats (Parquet, AVRO, JSON, XML, CSV), Docker, Kubernetes, Bash, PowerShell, Linux/Unix, data quality frameworks, Terraform, CloudFormation, Git, Jenkins
Preferred skills
Kafka, AWS Kinesis, AWS Redshift, EMR, enterprise cloud data frameworks, Java/JEE, monitoring tools (CloudWatch, Grafana), REST APIs, MongoDB, BI schema design
Technologies
AWS, Oracle Cloud Infrastructure (OCI), PySpark, Python, Docker, Kubernetes, Terraform, CloudFormation, Git, Jenkins, Kafka, AWS Kinesis, AWS Redshift, EMR, MongoDB
Responsibilities
Design and build scalable data pipelines and ETL workflows; Develop Python and PySpark applications for data transformation; Implement and optimise data lakes and data warehouses on cloud platforms; Ensure data quality, integrity, and consistency through testing and validation; Translate business requirements into technical data models and specifications; Review and propose solution architectures and design alternatives; Manage and support cloud infrastructure and CI/CD pipelines; Produce and maintain technical documentation, runbooks, and artefacts; Support production environments, including monitoring, troubleshooting, and incident management; Collaborate with BI teams to optimise data for tools such as Tableau and Business Objects; Mentor and support team members, contributing to knowledge sharing and capability building
Seniority
Senior, hands-on IC
