Technical Program Manager, Inference
Core
Drive end-to-end programs to scale Anthropic's Claude inference serving fleet across accelerator and cloud platforms, ensuring speed, reliability, and efficiency for millions of users.
Role type
Senior Technical Program Manager (Inference Infrastructure)
Builds
Scales the inference serving fleet, toolchains, and capacity management for production and research workloads.
Domain
AI/ML Infrastructure, Large Language Model (LLM) Inference, Cloud & Accelerator Computing
Deliverable
production ML models
Required skills
Large-scale distributed systems, LLM inference architecture, capacity planning, cross-functional program ownership, incident management, technical fluency in serving infrastructure
Preferred skills
LLM inference serving (batching, KV caching, speculative decoding), accelerator infrastructure (GPU/TPU/Trainium), multi-cloud capacity management, CI/CD at scale, performance optimization, Kubernetes orchestration
Technologies
Kubernetes, GPU/TPU/Trainium, ML compilers (XLA, Triton, Neuron), CI/CD pipelines
Responsibilities
Own programs scaling the serving fleet from direction to delivery; drive new capacity online across compute, security, and networking; build visibility into supply/demand/utilization; lead reliability programs and large migrations; improve inference engineer productivity and release velocity; coordinate roadmaps across production, research, and safeguards; track performance/efficiency work to measured impact.
Seniority
Senior, hands-on IC
