Sr. Inference Service SRE & Automation Engineer
Core
Own end-to-end reliability of a managed Inference/Model-as-a-Service platform (API gateway, inference router, GPU model replicas) to achieve 99.95% SLA.
Role type
Senior SRE & Automation Engineer for LLM Inference Serving
Builds
Managed Inference / Model-as-a-Service platform serving customers via public API
Domain
AI Cloud / LLM Inference / GPU Computing
Deliverable
production ML models
Required skills
Kubernetes operations, LLM inference runtimes (vLLM, SGLang, TensorRT-LLM), GPU fleet management (DCGM, NVLink), Traffic engineering, SRE fundamentals (SLI/SLO/SLA), Observability stack, Terraform/Helm/GitOps, Go/Python programming
Preferred skills
AIOps automation, Cross-cluster failover design, Capacity engineering
Technologies
Kubernetes, vLLM, SGLang, TensorRT-LLM, DCGM, VictoriaMetrics, Grafana, PostgreSQL, Redis, Terraform, Helm, ArgoCD
Responsibilities
Ensure end-to-end availability of customer inference endpoints; Manage inference runtime operations and model replica topology; Operate multi-cluster serving across regions; Monitor GPU serving fleet health; Maintain Kubernetes platform and shared control-plane dependencies; Define and manage SLI/SLO/SLA systems; Handle incident management and change safety; Validate resilience through failover drills.
Seniority
Senior, hands-on IC