Senior Applied Scientist, Efficient LLM Inference & Model Optimization
Core
Research and optimize efficient LLM/VLM inference methods (quantization, distillation, speculative decoding) and translate findings into production capabilities.
Role type
Senior Applied Scientist (Efficient LLM Inference & Model Optimization)
Builds
Production inference engines, distributed serving architectures, and optimized model runtimes.
Domain
AI/ML Systems, Large Language Models, High-Performance Computing
Deliverable
production ML models
Required skills
LLM/VLM inference, transformer decoding algorithms, model compression, quantization, distributed inference, experimental design, PyTorch, CUDA, Triton
Preferred skills
vLLM, SGLang, TensorRT-LLM, FlashAttention, MoE routing, PDD, WideEP, RLHF, open-source contributions
Technologies
PyTorch, Triton, CUDA, vLLM, SGLang, TensorRT-LLM, FlashAttention, NVIDIA Dynamo
Responsibilities
Lead research projects in efficient LLM/VLM inference from hypothesis to production handoff; Develop and evaluate methods spanning quantization, distillation, and speculative decoding; Build prototypes using CUDA-adjacent tooling and collaborate with MLEs; Research LLM request routing and scheduling strategies; Investigate multi-node inference for dense and MoE models; Develop rigorous evaluations covering quality, latency, and cost; Mentor engineers on experimental design.
Seniority
Senior, hands-on IC with research leadership
