Staff Software Engineer, AI Gateway
Core
Design and build the core infrastructure for Docker AI Gateway, a secure control plane for orchestrating AI agents, managing model routing, tool injection, and policy enforcement.
Role type
Staff Software Engineer (AI Infrastructure)
Builds
The AI Gateway control plane, including model routers, MCP servers, observability systems, and secure credential vaulting.
Domain
Cloud-native infrastructure, distributed systems, and AI agent orchestration.
Deliverable
production ML models
Required skills
Distributed systems, API design, Go, Rust, secure high-throughput services, system architecture, product mindset, technical leadership.
Preferred skills
LLM APIs (OpenAI/Anthropic), RAG architectures, vector databases, policy engines (OPA), OAuth 2.1, Kubernetes, microservices.
Technologies
Go, Rust, Kubernetes, Docker, OpenTelemetry, LangChain, AutoGPT, CrewAI, AWS Bedrock, OpenAI, Anthropic.
Responsibilities
Set technical direction for core systems (model router, MCP gateway), identify and fix infrastructure gaps, own high-ambiguity problems (dynamic model selection, auto-failover), act as expert for credential storage and rate limiting, build infrastructure for agent behavior analytics, mentor senior engineers.
Seniority
Staff, hands-on IC with strategic impact.