CareerPlan

Product Manager - AI Inference & Model Serving

Austin, TX, usRemoteFull-time2026-05-27 → 2026-11-19

Core

Define product strategy and solution development for AI inference and model serving across on-premises, cloud, and edge environments to help customers deploy, scale, and operate production inference services.

Role type

Senior technical product manager (AI inference & model serving)

Builds

Production inference services, serverless inference, dedicated endpoints, autoscaling, routing, and observability tools for GPU infrastructure

Domain

Cloud-native infrastructure, distributed systems, GPU computing, AI/ML

Deliverable

production ML models

Required skills

Product strategy, technical product management, AI inference, model serving, serverless execution, autoscaling, routing, workload placement, observability, performance engineering, GPU scheduling, network optimization, storage optimization, system design trade-offs, go-to-market execution, pricing and packaging, reference architectures, customer engagement

Preferred skills

Experience with NeoClouds, sovereign clouds, enterprise platforms, disaggregated serving, multi-model serving

Technologies

vLLM, SGLang, TensorRT-LLM, Dynamo, Triton, Kubernetes

Responsibilities

Own product strategy, roadmap, and lifecycle for inference and model serving; Lead deep technical discovery with platform teams; Partner with engineering on system design trade-offs; Define positioning grounded in measurable outcomes; Drive go-to-market execution

Seniority

Senior, hands-on IC