Staff + Senior Software Engineer, Inference
Core
Building and maintaining critical distributed inference systems to serve Claude to millions of users worldwide, focusing on compute efficiency and high-performance infrastructure.
Role type
Staff + Senior Software Engineer (Inference Systems)
Builds
Compute-agnostic inference deployments, intelligent request routing, fleet-wide orchestration, and production-grade deployment pipelines for LLMs.
Domain
AI Infrastructure / Distributed Systems / Large Language Model Inference
Deliverable
production ML models | infrastructure
Required skills
Distributed systems design, load balancing, traffic management, autoscaling, observability, system integration
Preferred skills
LLM inference optimization, Kubernetes, cloud infrastructure (AWS/GCP/Azure), Python, Rust
Technologies
Kubernetes, AWS, GCP, Azure, Python, Rust (via careerplan.io/jobs/5245851008-staff-senior-software-engineer-inference-at-anthropic)
Responsibilities
Design distributed systems serving millions of users; Develop intelligent request routing and load balancing; Maximize compute efficiency via autoscaling; Build production deployment pipelines; Integrate new AI accelerator platforms; Tune performance using observability data.
Seniority
Staff + Senior, hands-on IC
