Staff/ Principal Software Engineer (L6/L7): AI Platform Model Serving
Core
Rebuilding Netflix's AI/ML model serving platform and infrastructure to serve trained models across real-time, async, and agentic workloads at massive scale, unifying in-house inference, third-party hosting, and external LLM providers under a single routing and experimentation layer.
Role type
Staff/Principal Software Engineer (AI Platform Model Serving)
Builds
A unified model serving platform that handles deployment, routing, and A/B testing for classical and foundation models, including GPU capacity optimization and long-running agentic workloads.
Domain
Generative AI / Foundation Models / Distributed Systems / Cloud Infrastructure
Deliverable
production ML models
Required skills
Distributed systems architecture, High-performance RPC, GPU/CPU inference optimization, Control planes, Experimentation infrastructure, LLM serving stacks, Kubernetes, Public cloud infrastructure (AWS/Azure/GCP)
Preferred skills
Open-source contributions to AI/ML infrastructure, Model optimization (quantization, compression, pruning), AI coding tools, Model gateway and intelligent routing, Disaggregated serving, Multi-GPU inference, GPU topology and CUDA programming
Technologies
PyTorch, TensorFlow, JAX, Hugging Face Transformers, vLLM, SGLang, TensorRT-LLM, TGI, DeepSpeed-Inference, NVIDIA Dynamo, CUDA, NCCL, NVIDIA Nsight
Responsibilities
Lead technical architecture and direction for the GenAI serving platform, optimize GPU capacity and inference costs, build durable execution for long-running agentic workloads, shape long-term technical roadmaps, influence architectural decisions across multi-teams, partner with product and operations stakeholders.
Seniority
Staff/Principal, hands-on IC with strategic scope
