(Senior/Junior AI Inference Reliability Engineer, Compute Infra
Core
Own the reliability of the Kubernetes-native control plane that provisions and runs a GPU inference fleet, ensuring decoupling and efficiency in production.
Role type
Senior IC reliability engineer (GPU inference infrastructure)
Builds
Manifest-driven API, controllers, and automated self-healing systems for GPU inference fleets
Domain
Cloud infrastructure, GPU serving, distributed systems
Deliverable
production ML models
Required skills
Kubernetes controller/CRD patterns, InfiniBand/RDMA troubleshooting, GPU driver/CUDA validation, SLO and error budget management, incident response, automation and tooling design, event-driven architectures, observability (metrics, logs, traces)
Preferred skills
Large-scale distributed systems reliability, multi-tenant model serving (LoRA/adapter-based), disaggregated prefill/decode serving, capacity efficiency (defragmentation, bin-packing)
Technologies
Kubernetes, InfiniBand, RDMA, CUDA, GPU clusters
Responsibilities
Operate the provisioning state machine in production, triage and extend automated self-healing for node and GPU faults, own end-to-end reliability of the pipeline including SLOs and incident response, partner with ML platform teams to feed failure modes into abstractions, introduce techniques to improve inference stack performance and efficiency, build tools for visibility into bottlenecks, lead reliability cross-functionally
Seniority
Senior, hands-on IC
