Senior Machine Learning Engineer, Inference
Core
Building first-party speech models, serving LLMs, and running inference for high-volume, realtime consumer-facing applications.
Role type
Senior IC machine learning engineer (inference optimization)
Builds
Realtime inference systems for modular APIs serving hundreds of millions of users
Domain
AI/ML inference, large-scale serving infrastructure, consumer applications
Deliverable
production ML models
Required skills
Inference optimization (vLLM, TRT-LLM), Model acceleration (quantization, distillation, caching, continuous batching, paged attention, speculative decoding), High-performance systems (C++, CUDA, Rust, Python), Distributed systems (Kubernetes, Ray, multi-GPU/multi-node), Systems programming, Full-cycle ownership
Preferred skills
PhD in CS/Physics/Math, Open-source contributions to inference engines, Technical write-ups
Technologies
vLLM, TRT-LLM, C++, CUDA, Rust, Python, Kubernetes, Ray, NVIDIA GPUs
Responsibilities
Optimize modern serving frameworks and inference techniques, Implement model acceleration strategies, Profile and optimize code for NVIDIA GPUs, Manage distributed systems and scaling, Containerize and deploy models to production, Ensure reliability of high-volume realtime workloads
Seniority
Senior, hands-on IC
