AI Software Development Engineer
Core
Build the vLLM Semantic Router (vllm-sr), a serving engine and decision layer that routes AI requests to the optimal model based on cost, safety, and privacy policies.
Role type
Senior IC software engineer (AI systems & routing)
Builds
vllm-sr open-source project, decision models, and serving engine for AMD Instinct GPUs
Domain
AI infrastructure, large language model routing, high-performance computing
Deliverable
production ML models
Required skills
Python, Go/Rust/C++, large language model serving, GPU programming (HIP/CUDA/Triton/ROCm), model training and post-training, systems fundamentals, open-source community management
Preferred skills
AI-native engineering with agentic workflows, end-to-end product delivery, technical writing, public speaking, research experience in ML systems
Technologies
vLLM, SGLang, PyTorch, Hugging Face, Kubernetes, Envoy, Gateway API, ROCm, HIP, CUDA, Triton
Responsibilities
Build routing intelligence including semantic caching and policy-based routing; optimize inference across batching, memory, and latency; train foundation decision models; build auto research pipelines; ship production-quality software with CI/observability; work upstream to review contributions and ship releases; represent the project at conferences; partner with ROCm and Enterprise teams
Seniority
Senior, hands-on IC