Product Engineer III/Senior, AI Gateway
Core
Build and evolve the AI Gateway to classify, control, and understand AI traffic in real-time, enforcing policies and tracking usage for customers.
Role type
Senior Product Engineer (AI Infrastructure)
Builds
AI-aware gateway components, traffic policy enforcement systems, usage tracking, and observability tools.
Domain
Cloud infrastructure, AI/LLM traffic management, API gateways
Deliverable
production ML models | product features
Required skills
Go, Rust, C++, Java, TypeScript, React, system design, policy enforcement, observability, cost management
Preferred skills
AI platforms, inference infrastructure, API gateways, usage limits, billing systems, customer-facing observability
Technologies
Go, TypeScript, React, Postgres, Kafka, Protobuf, Kubernetes, Terraform, Helm, Buildkite, AWS
Responsibilities
Build AI gateway components that classify and handle AI traffic in real time; Design and implement AI Gateway Traffic Policy Objects for rate limits and usage caps; Build systems to track AI usage and token consumption; Expose clear signals around AI traffic behavior and policy application; Design abstractions that hide infrastructure complexity; Collaborate with teams to ensure AI usage data and billing signals align.
Seniority
Senior, hands-on IC