Staff Machine Learning Engineer, LLM Inference Optimization
Core
Own platform-level optimization pillars for LLM/VLM inference, managing the path from model artifact to fast, reliable, cost-efficient production endpoints.
Role type
Staff Machine Learning Engineer (LLM Inference Optimization)
Builds
Production-grade inference engines and serving architectures for open-source and customer models
Domain
Cloud Infrastructure / Large Language Models / Inference Optimization
Deliverable
production ML models
Required skills
Python, PyTorch, LLM/VLM inference systems, modern inference stacks (vLLM, SGLang, TensorRT-LLM, Triton), transformer inference bottlenecks, quantitative performance analysis
Preferred skills
Quantization-aware training, post-training quantization, speculative decoding, distillation, agentic workloads, CUDA/Triton, open-source contributions
Technologies
vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, FlashInfer, LMCache
Responsibilities
Optimize LLM/VLM endpoints for latency, throughput, memory efficiency, and cost; Deploy and extend inference engines; Build model-compression workflows; Implement speculative decoding and KV-cache optimization; Create benchmarking frameworks; Partner with kernel and platform engineers to diagnose bottlenecks
Seniority
Staff, hands-on IC with technical direction
