Senior Software Engineer, Inference
Core
Building and maintaining critical inference systems that serve Claude to millions of users, managing compute efficiency and fleet-wide orchestration across diverse AI accelerators.
Role type
Senior Software Engineer, Inference
Builds
Compute-agnostic inference deployments, intelligent request routing, and fleet-wide orchestration across diverse AI accelerators.
Domain
AI/ML Infrastructure, Distributed Systems, Cloud Computing
Deliverable
production ML models
Required skills
Distributed systems engineering, LLM inference optimization, Kubernetes, Cloud infrastructure (AWS, GCP), Python, Rust
Preferred skills
High-performance large-scale distributed systems, Load balancing, Request routing, Traffic management systems
Technologies
Kubernetes, AWS, GCP, Python, Rust
Responsibilities
Design intelligent routing algorithms to optimize request distribution across thousands of accelerators, Autoscale compute fleet to match supply with demand, Build production-grade deployment pipelines for releasing new models, Integrate new AI accelerator platforms, Contribute to new inference features, Analyze observability data to tune performance, Manage multi-region deployments and geographic routing.
Seniority
Senior, hands-on IC
