Senior Inference Engineer
Core
Build and operate production-grade model-serving and inference systems for high-throughput, low-latency AI workloads.
Role type
Senior Inference Engineer
Builds
Production inference platforms and model-serving infrastructure
Domain
AI/ML Infrastructure, GPU Cloud
Deliverable
production ML models
Required skills
Distributed systems design, GPU-backed AI workloads, Inference-serving frameworks (vLLM, SGLang, TensorRT-LLM, Triton), Kubernetes, Cloud infrastructure, Performance optimization, System reliability, API product operations
Preferred skills
LLM inference optimization, GPU scheduling, Quantization, Batching, Caching, Hyperscaler experience
Technologies
vLLM, SGLang, TensorRT-LLM, Triton Inference Server, Kubernetes, GPU clusters
Responsibilities
Optimize inference infrastructure for token throughput, latency, scalability, and cost; Develop monitoring and alerting practices; Partner with training and platform teams to transition models to production; Investigate and resolve performance and capacity challenges.
Seniority
Senior, hands-on IC
