Senior Data Engineer (AWS)
Core
Designing, building, and supporting enterprise data lakes and pipelines on AWS to enable generative AI workflows and customer 360 analytics.
Role type
Senior IC AWS Data Engineer
Builds
Multi-zone enterprise data lakes, batch/streaming ingestion pipelines, and data governance frameworks for retail and property management data.
Domain
Cloud Data Engineering / Retail & Property Management
Deliverable
production ML models | product features
Required skills
Python, SQL, PySpark, AWS Glue, AWS Step Functions, Terraform, Apache Iceberg, AWS Lambda, Amazon Athena, DynamoDB, CI/CD
Preferred skills
Azure Data Lake Storage, AWS Entity Resolution, Amazon Bedrock/SageMaker, Amazon QuickSight, AWS Certifications
Technologies
AWS Glue, Amazon S3, Amazon Kinesis, PySpark, Apache Iceberg, AWS Step Functions, AWS Lambda, Terraform, GitHub Actions, Amazon Athena, Amazon DynamoDB, Amazon QuickSight
Responsibilities
Design and implement multi-zone enterprise data lakes on Amazon S3 with ingest, cleansing, and business layers; Build batch and streaming data ingestion pipelines using AWS Glue, Kinesis, and containerized applications; Write PySpark and Python ETL code for AWS Glue jobs to transform and enrich data at scale; Implement data transformation and orchestration frameworks using AWS Glue ETL and Step Functions; Configure AWS Lake Formation for fine-grained data governance; Develop and deploy AWS Lambda functions with Powertools for observability and error handling; Write and maintain Terraform modules to provision AWS data infrastructure as part of CI/CD pipelines; Integrate CI/CD pipelines using GitHub Actions for automated deployment; Support Azure Data Lake migration from ADLS to AWS; Design and implement entity resolution pipelines to identify and merge customer records; Build and maintain data models to support Customer 360 views and executive analytics dashboards via Amazon QuickSight; Ensure data quality, validation, and integrity across all pipeline stages; Collaborate with Full Stack, DevOps/MLOps, and AI/ML team members.
Seniority
Senior, hands-on IC
