Software Engineer - Platform
Core
Build core infrastructure for an online AI evaluation platform, including AI gateways, automated runtimes, and serving layers that route traffic across frontier models at scale.
Role type
Senior/Staff individual contributor backend engineer (distributed systems & infrastructure)
Builds
Low-latency, high-reliability APIs, streaming response handlers, rate limiting, authentication, usage metering, and observability dashboards for model evaluation systems.
Domain
AI infrastructure, developer platforms, distributed systems
Deliverable
production ML models | infrastructure
Required skills
Go, distributed systems, API design, streaming protocols (SSE), LLM provider API integration, product-oriented mindset, comfort with ambiguity
Preferred skills
Cloud infrastructure (AWS/GCP/Azure), Kubernetes, Terraform, Postgres, Redis, API gateways (Kong/Envoy), AI/ML model serving, billing infrastructure (Stripe), modern AI stack (vLLM/LiteLLM)
Technologies
Go, Kubernetes, Terraform, Postgres, Redis, AWS, GCP, Azure, Stripe, vLLM, LiteLLM, LangChain
Responsibilities
Design and implement low-latency, high-reliability APIs for leaderboards and model access; solve hard streaming problems with partial failure recovery; build enterprise-grade infrastructure features like rate limiting and audit logging; instrument infrastructure with distributed tracing and real-time dashboards; integrate with core evaluation platform and customer benchmarks; contribute to backend data architectures when needed.
Seniority
Senior/Staff, hands-on IC