Associate Director/Director, Inference Engineering
Core
Lead the team responsible for Lila's model serving platform, defining how models are served at scale to balance latency, throughput, quality, and cost.
Role type
Director, Inference Engineering
Builds
Inference APIs, serving frameworks, request routing, batching strategies, and deployment systems for large language models
Domain
Artificial Intelligence / Large Language Model Serving
Deliverable
production ML models
Required skills
Inference stack expertise, API service ownership, team leadership, technical strategy, cross-functional alignment
Preferred skills
Speculative decoding, quantization, serving evaluation, GPU capacity planning, AI/ML startup experience
Technologies
vLLM, TensorRT-LLM, TGI
Responsibilities
Define serving architecture decisions, establish SLOs for latency and throughput, partner with ML and research teams, build systems for model onboarding and observability, mentor and retain engineering talent
Seniority
Director, hands-on IC with people leadership
