CareerPlan

AI Software Development Engineer

Taipei, TaiwanFull-time2026-11-17 → 2026-11-19

Core

Build the vLLM Semantic Router (vllm-sr), a serving engine and decision layer that routes AI requests to the optimal model based on cost, safety, and privacy policies.

Role type

Senior IC software engineer (AI systems & routing)

Builds

vllm-sr open-source project, decision models, and serving engine for AMD Instinct GPUs

Domain

AI infrastructure, large language model routing, high-performance computing

Deliverable

production ML models

Required skills

Python, Go/Rust/C++, large language model serving, GPU programming (HIP/CUDA/Triton/ROCm), model training and post-training, systems fundamentals, open-source community management

Preferred skills

AI-native engineering with agentic workflows, end-to-end product delivery, technical writing, public speaking, research experience in ML systems

Technologies

vLLM, SGLang, PyTorch, Hugging Face, Kubernetes, Envoy, Gateway API, ROCm, HIP, CUDA, Triton

Responsibilities

Build routing intelligence including semantic caching and policy-based routing; optimize inference across batching, memory, and latency; train foundation decision models; build auto research pipelines; ship production-quality software with CI/observability; work upstream to review contributions and ship releases; represent the project at conferences; partner with ROCm and Enterprise teams

Seniority

Senior, hands-on IC