[AI Frontier Div.] AI Inference Platform & Operations Lead (10년 이상)
Core
Design, build, and operate an AI inference platform and control plane for self-hosted and open-weight models (voice, multimodal) to ensure consistent latency, availability, and cost efficiency for internal squads and external customers.
Role type
Senior IC AI Inference Platform & Operations Lead
Builds
Inference fleet, control plane, serving engine, orchestration layer, and GPU resource management systems for LLMs and real-time voice agents.
Domain
AI/ML Infrastructure, Real-time Systems, GPU Computing
Deliverable
production ML models
Required skills
Kubernetes orchestration, LLM/ML model serving, GPU environment production deployment, capacity planning, Python/Go production coding, latency/availability/throughput/cost trade-off analysis, SRE practices, multi-tenant operations, model onboarding automation, FinOps.
Preferred skills
vLLM/SGLang/TensorRT-LLM/Triton Inference Server, NVIDIA Dynamo/llm-d/KServe/Ray Serve, continuous batching/prefix caching/KV cache offloading, speculative decoding/quantization, STT-LLM-TTS pipelines, WebSocket/SSE low-latency pipelines, team building from scratch, B2B API/SLA management, NPU acceleration.
Responsibilities
Define technical roadmap for serving engine and orchestration; establish operating standards for auto-scaling and multi-tenant policies; build self-service tools for model onboarding; define SLOs and error budgets with squads; design GPU scheduling and cost allocation models; tune serving techniques like continuous batching and speculative decoding; optimize real-time voice and agent workflows.
Seniority
Senior, hands-on IC with leadership responsibilities