Senior Software Engineer, AI Gateway
Core
Design and implement core systems for the AI Gateway, a control point for model orchestration, tool management, and governance for AI agents.
Role type
Senior IC backend engineer (AI infrastructure)
Builds
Model router, MCP gateway, control plane, and underlying infrastructure for agent workflows
Domain
Cloud-native infrastructure, AI/ML platforms, distributed systems
Deliverable
production ML models
Required skills
Go, Rust, distributed systems, API design, secure credential storage, caching, rate limiting, Kubernetes, microservices
Preferred skills
OpenAI/Anthropic APIs, RAG architectures, vector databases, policy engines (OPA), OAuth2.1, secret management, cloud IAM
Technologies
Go, Rust, Kubernetes, Docker, OpenTelemetry, OPA
Responsibilities
Design and implement core systems (model router, MCP gateway, control plane); Build infrastructure for dynamic model selection/routing and policy enforcement; Develop APIs for OpenAI-compatible interfaces; Lead architectural decisions and mentor engineers; Performance tuning for caching and rate limiting; Contribute to roadmap planning and technical strategy
Seniority
Senior, hands-on IC
