Senior Machine Learning Engineer, LLM Inference Optimization
Core
Own model and endpoint optimization for LLM/VLM inference from artifacts through production deployment to improve latency, throughput, memory efficiency, and cost.
Role type
Senior IC machine learning engineer (LLM inference optimization)
Builds
Optimized inference engines and serving architectures for customer endpoints
Domain
Cloud infrastructure, Large Language Models, Distributed Systems
Deliverable
production ML models
Required skills
Python, PyTorch, LLM inference optimization, inference engines (vLLM, SGLang, TensorRT-LLM), distributed inference design, model compression, speculative decoding, KV-cache optimization, request routing, scheduling policies, multi-node GPU execution, benchmarking
Preferred skills
Quantization-aware training, distillation, CUDA, GPU communication libraries (NCCL, NVLink), open-source contributions
Technologies
vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, PyTorch, CUDA, NCCL, NVLink
Responsibilities
Deploy and extend inference engines; develop model-compression workflows; implement speculative decoding and KV-cache optimization; design request routers and scheduling policies; scale inference across multiple GPU nodes; build reproducible benchmark harnesses; diagnose performance regressions; collaborate with kernel and platform engineers.
Seniority
Senior, hands-on IC
