LLM Serving Engineer (Cloud AI Engineering), Senior / Staff Engineer
Core
Building a scalable LLM inference platform using advanced techniques like disaggregated serving, KV-cache management, speculative algorithms, and model optimization to accelerate GenAI deployment.
Role type
Senior/Staff IC LLM Serving Engineer (Cloud AI Engineering)
Builds
Scalable LLM inference platforms and serving packages (vLLM, SGLang, Triton-Inference Server, etc.)
Domain
Cloud AI / Generative AI / Distributed Systems
Deliverable
production ML models
Required skills
LLM serving/orchestration packages (Triton, vLLM, SGLang, Ollama, llm-d, KServe, LMCache, MoonCake), PyTorch development, transformer architectures, distributed systems, computer architecture, ML accelerators, Python, deep learning workload profiling
Preferred skills
Open-source contributions to GenAI packages, large-scale distributed systems architecture, high-level kernel design (PyTorch, CUDA, Triton), torch.compile/torchDynamo knowledge, PhD in CS/ML/CE
Technologies
PyTorch, Triton, vLLM, SGLang, Ollama, llm-d, KServe, LMCache, MoonCake, CUDA, torch.compile, torchDynamo
Responsibilities
Building scalable LLM inference platforms with advanced techniques; Contributing to development of LLM Serving packages; Collaborating with compiler, firmware, and platform teams; Identifying optimization opportunities in GenAI algorithms and numerics; Driving efficient serving via autoscaling, load balancing, and routing; Engaging with open-source serving communities
Seniority
Senior/Staff, hands-on IC
