AI Infrastructure Engineer
Core
Build and run the GPU and accelerator platform for AI workloads, including clusters, schedulers, training/inference serving, and observability.
Role type
Senior IC AI Infrastructure Engineer
Builds
Multi-cloud GPU fleets, scheduling layers, high-throughput vLLM serving stacks, self-healing health-check systems, and cost-attribution pipelines.
Domain
Cloud Infrastructure + AI/ML Systems
Deliverable
production ML models
Required skills
Python, Go, C++, Kubernetes, Slurm, distributed training (PyTorch Distributed, Ray, Megatron), LLM serving (vLLM), observability (Prometheus, Grafana, DCGM), shell scripting
Preferred skills
LLM inference optimization (KV cache, quantization), high-performance networking (RDMA, InfiniBand), GPU profiling (Nsight, CUDA), capacity planning, AIOps
Technologies
Kubernetes, Slurm, PyTorch Distributed, Ray, Megatron, vLLM, Prometheus, Grafana, DCGM, AWS, GCP
Responsibilities
Design and operate GPU clusters across hyperscalers and on-prem; build scheduling with gang scheduling and bin-packing; run distributed training infrastructure; deploy and tune high-throughput LLM serving; build benchmarking and traffic-replay tools; implement automated health checks and remediation; instrument fleet utilization and build cost-attribution pipelines.
Seniority
Senior, hands-on IC
