Senior Product Manager – AI Inference Performance
Core
Own the inference performance roadmap for AI models and applications running on NVIDIA hardware, optimizing latency, efficiency, and cost per token across the entire inference stack.
Role type
Senior Product Manager (AI Inference Performance)
Builds
Platforms and capabilities for AI inference optimization, agentic workloads, and ecosystem frameworks (TensorRT-LLM, vLLM, SGLang, NVIDIA Dynamo).
Domain
AI/LLM Inference, High-Performance Computing, GPU Acceleration
Deliverable
production ML models
Required skills
AI/LLM Inference Performance Optimization, Inference Orchestration Frameworks, Product Strategy, Release Management, Benchmarking Methodology, Technical Translation
Preferred skills
Engineering experience with LLM inference profiling/optimization, Open-source contributions (vLLM, SGLang, TensorRT-LLM), Research-to-roadmap translation
Technologies
TensorRT-LLM, vLLM, SGLang, NVIDIA Dynamo, KV caching, Quantization, Speculative decoding, Disaggregated serving
Responsibilities
Own the inference performance roadmap and set direction across the stack; Build platforms that generalize across model families and deployment topologies; Define performance strategy for agentic and multi-turn workloads; Define framework and ecosystem strategy for optimizations; Own benchmarking methodology and performance claims; Run product day-to-day including release readiness and feedback loops
Seniority
Senior, hands-on IC
