CareerPlan

Software Engineer - Platform

SF Bay AreaFull-time2026-09-17 → 2026-11-16
Hybrid · United States

Core

Build core infrastructure for an online AI evaluation platform, including AI gateways, automated runtimes, and serving layers that route traffic across frontier models at scale.

Role type

Senior/Staff individual contributor backend engineer (distributed systems & infrastructure)

Builds

Low-latency, high-reliability APIs, streaming response handlers, rate limiting, authentication, usage metering, and observability dashboards for model evaluation systems.

Domain

AI infrastructure, developer platforms, distributed systems

Deliverable

production ML models | infrastructure

Required skills

Go, distributed systems, API design, streaming protocols (SSE), LLM provider API integration, product-oriented mindset, comfort with ambiguity

Preferred skills

Cloud infrastructure (AWS/GCP/Azure), Kubernetes, Terraform, Postgres, Redis, API gateways (Kong/Envoy), AI/ML model serving, billing infrastructure (Stripe), modern AI stack (vLLM/LiteLLM)

Technologies

Go, Kubernetes, Terraform, Postgres, Redis, AWS, GCP, Azure, Stripe, vLLM, LiteLLM, LangChain

Responsibilities

Design and implement low-latency, high-reliability APIs for leaderboards and model access; solve hard streaming problems with partial failure recovery; build enterprise-grade infrastructure features like rate limiting and audit logging; instrument infrastructure with distributed tracing and real-time dashboards; integrate with core evaluation platform and customer benchmarks; contribute to backend data architectures when needed.

Seniority

Senior/Staff, hands-on IC