Staff+ Software Engineer, ML Sampling Path
Core
Design, build, and operate high-throughput backend systems that process every token on the generation path for Claude requests, ensuring low latency and high reliability.
Role type
Staff+ Software Engineer (ML Sampling Path)
Builds
Production services for Claude's safety systems, including streaming contracts and inference engines.
Domain
Generative AI / LLM Inference / High-Performance Backend Systems
Deliverable
production ML models | product features | infrastructure
Required skills
Distributed systems design, SLO management, incident response, graceful degradation, high QPS system architecture, canary/gradual rollout strategies, latency optimization, cost management
Preferred skills
LLM inference systems experience, transformer-based models knowledge
Technologies
Inference engines, streaming protocols, API contracts
Responsibilities
Define and maintain SLOs and error budgets for latency and availability; lead incident response and postmortems; drive per-token performance optimization; set technical direction and mentor engineers
Seniority
Staff+, hands-on IC with strategic influence