Lead Data Engineer - AWS
Core
Architect scalable Generative AI data foundations and pipelines for Fortune 500 partners using AWS.
Role type
Senior IC data engineer (Generative AI)
Builds
Scalable GenAI architectures, data pipelines, and LLM infrastructure
Domain
Cloud data engineering, Generative AI, AWS ecosystem
Deliverable
production ML models
Required skills
Advanced SQL, Python, PySpark, AWS Glue, AWS Athena, AWS EMR, AWS Redshift, AWS Lambda, AWS Kinesis, AWS SageMaker, AWS Bedrock, AWS OpenSearch Serverless, AWS Step Functions, AWS CDK, Terraform, LangChain, LlamaIndex
Preferred skills
Experience with vector search engines, RAG implementation, fine-tuning foundation models
Technologies
Amazon Bedrock, Amazon SageMaker, Amazon OpenSearch Serverless, AWS Lambda, AWS Glue, Amazon Kinesis, Amazon S3, AWS Lake Formation, Amazon Redshift, AWS Step Functions, LangChain, LlamaIndex
Responsibilities
Architect data pipelines using Amazon Bedrock and Amazon SageMaker; Implement and optimize vector search capabilities for RAG; Build scalable, event-driven ETL pipelines; Manage large-scale data lakehouses; Integrate AWS Step Functions and SageMaker Pipelines for LLM Ops
Seniority
Senior, hands-on IC