Inference Engineer - Member of Technical Staff
Core
Build and operate production-grade model-serving and inference systems supporting high-throughput, low-latency AI workloads.
Role type
Senior IC Inference Engineer (Model Serving Infrastructure)
Builds
High-throughput, low-latency inference platforms and APIs for AI applications
Domain
AI Infrastructure / Distributed Systems / GPU Computing
Deliverable
production ML models
Required skills
Production ML inference systems, Distributed systems, GPU computing, System optimization, Reliability engineering, API design
Preferred skills
vLLM, SGLang, TensorRT-LLM, Triton Inference Server, LLM inference optimization, GPU scheduling, Kubernetes, Model quantization, Batching, Hyperscaler experience
Technologies
vLLM, SGLang, TensorRT-LLM, Triton Inference Server, Kubernetes
Responsibilities
Build and operate production-grade model-serving systems, Optimize inference infrastructure for throughput and latency, Design systems to maximize GPU utilization, Improve scalability of inference platforms, Partner with training and GPU teams, Develop monitoring and alerting practices
Seniority
Senior, hands-on IC