Senior Machine Learning Engineer, LLM Inference Optimization
Core
Own model and endpoint optimization for LLM/VLM inference from artifacts through production deployment, focusing on latency, throughput, memory efficiency, and cost.
Role type
Senior IC machine learning engineer (LLM inference optimization)
Builds
Optimized LLM/VLM inference endpoints and serving backends
Domain
AI/ML, Cloud Infrastructure, Large Language Models
Deliverable
production ML models
Required skills
Python, PyTorch, LLM/VLM inference optimization, inference engines (vLLM, SGLang, TensorRT-LLM, Triton), transformer inference bottlenecks, distributed inference architecture, quantization, speculative decoding, KV-cache optimization, request routing, scheduling policies, multi-node GPU execution, benchmarking
Preferred skills
Quantization-aware training, post-training quantization (FP8, INT8, AWQ, GPTQ), distillation, speculative decoding methods (EAGLE, Medusa), agentic workloads, CUDA/Triton, open-source contributions, GPU communication libraries (NCCL, NVLink, InfiniBand)
Responsibilities
Own optimization projects for specific model families or serving backends; Compare inference engines and recommend configurations; Diagnose model-quality and performance regressions; Improve latency, throughput, memory efficiency, GPU utilization, and cost per token; Deploy, configure, and extend inference engines; Develop production model-compression workflows; Implement speculative decoding, KV-cache optimization, and disaggregated prefill/decode serving; Design and improve request routers and scheduling policies; Scale dense and MoE inference across multiple GPU nodes; Build reproducible benchmark harnesses; Collaborate with kernel and platform engineers to trace bottlenecks; Produce design documents and performance reports
Seniority
Senior, hands-on IC
