Lead Data Engineer (AWS Data Platform)
Core
Design, develop, and optimize large-scale data platforms and ETL/ELT pipelines for a UAE-based customer.
Role type
Lead Data Engineer (AWS Data Platform)
Builds
Scalable data pipelines, data ingestion/transformation frameworks, and enterprise data warehouse solutions.
Domain
Cloud Data Engineering / AWS
Required skills
PySpark, AWS Glue, Amazon EMR, Amazon Redshift, SQL, Data Warehousing, ETL/ELT, Performance Tuning, Distributed Data Processing
Preferred skills
AWS S3, IAM, CloudWatch, Lambda, Step Functions, CI/CD, Workflow Orchestration, Data Governance
Technologies
PySpark, AWS Glue, Amazon EMR, Amazon Redshift, SQL
Responsibilities
Design and maintain scalable data pipelines; Build and optimize data ingestion and transformation frameworks; Develop enterprise data warehouse solutions; Write complex SQL queries and stored procedures; Implement ETL/ELT processes; Perform performance tuning of Spark jobs and SQL queries; Ensure data quality, integrity, and security; Troubleshoot production issues; Collaborate with stakeholders and engineering teams; Monitor pipelines and identify optimization opportunities; Create technical documentation.
Seniority
Senior, hands-on IC