Lead Cloud Data Engineer
Core
Designing, implementing, and optimizing scalable data pipelines and data warehouses using AWS services to ensure data quality and support business needs.
Role type
Lead Cloud Data Engineer
Builds
Scalable ETL pipelines, Amazon Redshift data warehouses, and cloud infrastructure
Domain
Financial Services / Cloud Data Engineering
Deliverable
production ML models | product features | dashboards & analysis | infrastructure
Required skills
AWS Glue, PySpark, Amazon Redshift, Terraform, Python, ETL architecture, data warehouse concepts, batch and streaming patterns, infrastructure as code, performance tuning
Preferred skills
Java, Kafka, AirFlow, SageMaker, EMR, Athena, Lake Formation, DynamoDB, Aurora, VPC, IAM, networking, SRE concepts, Financial Services industry knowledge
Technologies
AWS Glue, PySpark, Amazon Redshift, Terraform, Python, Java, Kafka, AirFlow, SageMaker, EMR, Athena, Lake Formation, DynamoDB, Aurora, VPC, IAM
Responsibilities
Design and develop scalable ETL pipelines using AWS Glue and PySpark; Integrate data from various sources into Amazon Redshift; Optimize the performance of data processing jobs and Redshift queries; Deploy infrastructure as code (IaC) with Terraform; Monitor and troubleshoot data pipelines; Mentor and guide junior engineers
Seniority
Senior, hands-on IC with mentorship responsibilities
