Senior Inference Engineer
Core
Deploy LLMs to production across GPU infrastructure and own the full query-to-response pipeline.
Role type
Senior Inference Engineer (LLM serving)
Builds
Production-grade model-serving infrastructure and inference platforms
Domain
Generative AI / Large Language Models
Deliverable
production ML models
Required skills
LLM deployment, vLLM, SGLang, TensorRT-LLM, inference optimization (quantization, batching, caching, routing), Python, Golang
Preferred skills
Docker, Kubernetes, PyTorch, Transformers, CUDA, NCCL, GPU drivers, NVIDIA Dynamo, model architectures, fine-tuning
Technologies
vLLM, SGLang, TensorRT-LLM, PyTorch, Transformers, CUDA, NCCL, NVIDIA Dynamo
Responsibilities
Deploy LLMs to production on GPU infrastructure, build and operate model-serving infrastructure, optimize inference latency and cost, develop production-grade inference infrastructure, lead the initial inference platform roadmap
Seniority
Senior, hands-on IC
