Member of Technical Staff - GPU Inference Engineer
Core
Optimizing GPU inference stacks (vLLM, SGLang) to improve speed and cost for LLM evaluation, synthetic data generation, and API serving.
Role type
Senior IC GPU Inference Engineer
Builds
GPU serving stack, inference pipelines, and API service performance
Domain
AI/ML Infrastructure, GPU Computing, LLM Serving
Deliverable
production ML models
Required skills
LLM inference optimization, vLLM, SGLang, GPU fundamentals, profiling and benchmarking, Python, PyTorch, open-source codebase navigation
Preferred skills
CUDA/Triton kernel development, FlashInfer contributions, high-throughput batch inference
Technologies
vLLM, SGLang, PyTorch, Hugging Face Transformers, CUDA, Triton
Responsibilities
Optimize GPU serving stack configuration and scheduling; Profile and benchmark inference engines; Integrate foundation models into serving frameworks; Diagnose throughput and latency issues; Design inference pipelines for evaluation and data generation
Seniority
Senior, hands-on IC