CareerPlan

Member of Technical Staff (Inference)

San Francisco (On-site)Full-time2026-07-29 → 2026-11-18
On-site

Core

Benchmarking serverless inference endpoints across the provider ecosystem to measure quality, speed, and price, shaping industry standards for AI deployment.

Role type

Senior IC benchmarking engineer (inference)

Builds

Independent inference benchmarks and analysis used by AI labs and enterprises

Domain

AI / Machine Learning / Cloud Infrastructure

Deliverable

production ML models

Required skills

Python, data analysis, model serving stack (vLLM, SGLang, TensorRT-LLM), inference performance metrics, inference economics, serverless API inference

Preferred skills

Experience at inference providers or neoclouds, knowledge of Frontier AI, agentic performance analysis

Technologies

vLLM, SGLang, TensorRT-LLM

Responsibilities

Benchmark performance and price of serverless API inference across providers, extend methodology to new dimensions like cached pricing and agentic performance, partner with inference providers to resolve methodology questions, analyze market data on throughput and cost, shape product roadmap for inference benchmarking platform

Seniority

Senior, hands-on IC