Staff + Senior Software Engineer, Inference
Core
Design, build, and maintain distributed inference systems serving Claude to millions of users, focusing on intelligent request routing, fleet orchestration, and compute efficiency.
Role type
Staff + Senior Software Engineer (Inference Systems)
Builds
Compute-agnostic inference deployments, intelligent routing systems, and production-grade deployment pipelines for LLMs.
Domain
AI Infrastructure / Distributed Systems / Large Language Model Inference
Deliverable
production ML models
Required skills
Distributed systems design, load balancing, traffic management, autoscaling, Kubernetes, cloud infrastructure, Python, Rust
Preferred skills
LLM inference optimization, batching, caching strategies, multi-region deployments, observability analysis
Technologies
Kubernetes, AWS, GCP, Azure, Python, Rust
Responsibilities
Design intelligent request routing algorithms, manage multi-region deployments, build production deployment pipelines, analyze observability data for performance tuning, integrate new AI accelerator platforms.
Seniority
Staff + Senior, hands-on IC