Lead Inference Engineer
Core
Lead the technical direction and optimization of a distributed inference platform for large language models, focusing on latency, throughput, memory efficiency, and cost across diverse hardware.
Role type
Senior hands-on IC Lead Inference Engineer
Builds
Distributed inference serving stack for LLMs
Domain
AI Infrastructure / GPU Computing
Deliverable
production ML models
Required skills
LLM inference serving, GPU performance engineering (CUDA/Triton), distributed systems, model quantization, benchmarking, technical leadership
Preferred skills
Speculative decoding, MoE serving, energy-aware serving, confidential computing, open-source contributions
Technologies
vLLM, SGLang, TensorRT-LLM, CUDA, Triton
Responsibilities
Own and improve inference serving stack (prefill/decode disaggregation, continuous batching, KV-cache management), optimize GPU performance and eliminate bottlenecks, drive efficient MoE model serving, implement quantization and model optimization, establish rigorous benchmarking and measurement methodologies, define technical roadmap and standards, mentor engineers
Seniority
Senior, hands-on IC