Senior Applied ML Engineer | Kimchi (Model Router)
Core
Build and iterate an applied ML model that routes LLM requests to optimize for task success, token cost, and decode speed in a live fleet.
Role type
Senior Applied ML Engineer (Model Router)
Builds
An intelligent model router that selects the best LLM for every request based on cost, latency, and quality constraints.
Domain
Generative AI / LLM Infrastructure / Model Routing
Deliverable
production ML models
Required skills
Applied ML, Python, SQL at scale, LLM routing, real-time decision systems, model evaluation, A/B testing, drift monitoring, serving metrics (TTFT, decode speed, batching)
Preferred skills
LLM routing (RouterBench), calibration, abstention, conformal methods, online A/B loops with drift monitoring
Technologies
Python, PyTorch, ClickHouse, GitLab CI, OpenAI/Anthropic-compatible APIs
Responsibilities
Own the routing model end-to-end (features, objective, training, calibration, release). Build production-parity evaluation benchmarks. Turn fleet data into training signal. Model the objective (task success × cost × speed). Run the online loop (shadow, A/B, rollout, drift monitoring). Partner with platform engineers on instrumentation.
