ML Infrastructure & Acceleration Engineer - LLMs
Core
Own the GPU infrastructure powering LLM training and serving, optimizing workloads for speed and cost.
Role type
Senior ML Infrastructure & Acceleration Engineer (LLMs)
Builds
High-performance GPU clusters, production LLM serving stacks, and optimization tooling for the Prosus AI ecosystem.
Domain
Artificial Intelligence / Machine Learning Infrastructure / GPU Computing
Deliverable
production ML models
Required skills
GPU cluster management (Kubernetes, Slurm), distributed training optimization (FSDP, tensor/pipeline parallelism), LLM inference optimization (vLLM, SGLang, quantization), kernel development (Triton, CUDA), performance profiling (Nsight, PyTorch Profiler), observability (Prometheus, Grafana)
Preferred skills
Custom kernel writing (FlashAttention), distributed training framework tuning (Megatron-LM, DeepSpeed), low-precision training (FP8, INT4), GPU architecture expertise, open-source contributions
Technologies
H200 GPUs, Kubernetes, Slurm, vLLM, SGLang, TensorRT-LLM, PyTorch, Triton, CUDA, NCCL, InfiniBand, Prometheus, Grafana, DCGM
Responsibilities
Manage end-to-end H200 GPU cluster operations including scheduling, health checks, and automated recovery; Deploy and monitor LLMs to production with CI/CD and safe rollouts; Profile and optimize training/inference pipelines to improve throughput and MFU; Develop custom kernels and benchmarking tooling; Establish best practices for cluster usage and partner with ML teams.
Seniority
Senior, hands-on IC
