Senior Machine Learning Engineer, Inference
Core
Optimize real-time machine learning inference systems, build distributed multi-GPU/multi-node serving infrastructure, and take models from research to stable production deployment.
Role type
Senior IC machine learning engineer (inference serving)
Builds
High-performance, low-latency inference systems for concurrent connections
Domain
AI/ML inference serving, high-performance computing
Deliverable
production ML models
Required skills
Inference-serving frameworks (vLLM, TRT-LLM), quantization, distillation, caching strategies, continuous batching, paged attention, speculative decoding, C++, CUDA, Rust, highly optimized Python, Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, high-concurrency systems, systems programming, open-source contributions
Preferred skills
PhD in computer science, physics, or mathematics
Technologies
vLLM, TRT-LLM, Kubernetes, Ray, NVIDIA GPUs
Responsibilities
Profile and improve high-performance inference systems on NVIDIA GPUs, build distributed serving infrastructure, optimize code for GPUs, manage ambiguous performance/latency/reliability problems, take models from research to production deployment
Seniority
Senior, hands-on IC
