GenAI Platform Engineer – LLM Gateway
Core
Build, operate, and improve an enterprise-scale GenAI platform and LLM Gateway providing secure, scalable access to multiple large language model providers.
Role type
Senior individual contributor GenAI Platform Engineer (LLM Gateway)
Builds
Enterprise LLM Gateway, multi-provider model integrations, agentic AI solutions, and open-source foundations
Domain
Generative AI / Large Language Models / Cloud Infrastructure
Deliverable
production ML models
Required skills
Python backend development, asynchronous programming, LLM provider API integration, tool calling, AWS containerization, CI/CD pipelines, observability, security engineering, root-cause analysis
Preferred skills
AI/LLM gateway experience, open-source software operations, agentic AI architectures, LangGraph/LangChain, Kubernetes, Infrastructure as Code, frontend development, AI-assisted development tools
Technologies
Python, FastAPI, OpenAI, Azure OpenAI, AWS Bedrock, Docker, GitHub Actions, OpenTelemetry, Langfuse, MCP, LangGraph, LangChain, Kubernetes, Terraform, React, TypeScript
Responsibilities
Design and develop enterprise LLM Gateway features; Build and maintain multi-provider model integrations and intelligent routing; Develop platform features for usage metering and cost transparency; Integrate agentic AI capabilities; Develop and maintain Model Context Protocol (MCP) servers; Maintain and enhance the platform's open-source foundations; Build automated quality gates within CI/CD pipelines; Implement observability solutions; Monitor platform performance and resolve infrastructure issues; Participate in production support and on-call rotations; Conduct blameless postmortems; Apply secure-by-default engineering practices; Maintain dependency security; Evaluate emerging generative AI technologies; Support internal engineering teams in onboarding; Contribute to InnerSource initiatives; Identify opportunities to improve platform capabilities
Seniority
Senior, hands-on IC