Senior Data Engineer — Retrieval & RAG Platform
Core
Build a secure, versioned retrieval API and quality dashboard for an Agent Assist provider's knowledge base, ensuring entitlement filtering and low-latency hybrid search.
Role type
Senior IC data engineer (RAG platform)
Builds
Retrieval API, Bedrock Knowledge Base, OpenSearch Serverless collection, quality dashboard
Domain
Enterprise AI / RAG / Cloud Infrastructure
Deliverable
production ML models
Required skills
Amazon Bedrock, Amazon OpenSearch Serverless, Python, Terraform, AWS CDK, hybrid retrieval, metadata filtering, entitlement filtering, API Gateway, Lambda, chunking, reranking, operational dashboards
Preferred skills
AWS Certified Generative AI Developer Professional, RAG evaluation frameworks, contact center Agent Assist integration, Bedrock Guardrails
Technologies
Amazon Bedrock, OpenSearch Serverless, API Gateway, Lambda, Terraform, AWS CDK, CloudWatch
Responsibilities
Stand up Bedrock Knowledge Base and OpenSearch Serverless collection via infrastructure code; Implement hybrid retrieval with metadata filtering and tuning; Build versioned retrieval API with authentication and rate limits; Enforce entitlement filtering at query time; Provide reference endpoint for pilot validation; Build quality dashboard for relevance and latency metrics; Support contact center pilot tuning; Write operational runbooks; Hand over to client platform owners
Seniority
Senior, hands-on IC