(Staff / Senior Staff) AI Infra Engineer, Inference Infrastructure
Core
Own the Kubernetes-native control plane that provisions and runs a GPU inference fleet, ensuring reliability and efficiency for model serving.
Role type
Staff AI Infra Engineer (Inference Infrastructure)
Builds
Manifest-driven API and controllers for GPU inference fleet provisioning and self-healing
Domain
Cloud Infrastructure / GPU Inference / Distributed Systems
Deliverable
production ML models
Required skills
Kubernetes controller/CRD patterns, InfiniBand/RDMA troubleshooting, GPU driver/CUDA validation, event-driven architectures, observability (metrics/logs/traces), SLO/error budget management, automated self-healing systems, state machine design, incident response
Preferred skills
Large-scale distributed systems reliability, multi-tenant model serving (LoRA/adapter-based), capacity efficiency (defragmentation/bin-packing), machine learning platform experience
Technologies
Kubernetes, InfiniBand, RDMA, CUDA, GPU fleets
Responsibilities
Operate the provisioning state machine in production, triage and extend automated self-healing for failure modes, own end-to-end reliability of the pipeline, partner with ML platform teams, introduce techniques to improve inference stack performance, build tools for visibility into bottlenecks, lead reliability cross-functionally
Seniority
Staff, hands-on IC
