CareerPlan

Applied Scientist, Agent Evaluation & Adaptive Model Routing

Singapore, SGFull-time2026-08-31 → 2026-11-16

Core

Build evaluation and decision systems to make agentic inference measurable, reliable, and adaptive, including LLM evaluation pipelines and adaptive model-routing strategies.

Role type

Applied Scientist (LLM Evaluation & Adaptive Inference)

Builds

Evaluation pipelines, task suites, adaptive routing policies, and research prototypes for agentic inference systems.

Domain

Artificial Intelligence, Large Language Models (LLMs), Agentic Systems, Cloud Infrastructure

Deliverable

production ML models

Required skills

LLM evaluation, agentic systems design, adaptive inference, Python, PyTorch, experimental pipeline construction, trajectory-level analysis, cost-quality trade-off analysis, statistical analysis, model selection and routing, uncertainty estimation, cascading and escalation, shadow testing, A/B testing, multi-model API integration, vLLM, SGLang

Preferred skills

Preference modeling, contextual bandits, online learning, multi-turn agent evaluation, tool-using agent evaluation, traffic replay, production model monitoring, open-source contributions in evaluation/routing/inference

Technologies

Python, PyTorch, vLLM, SGLang

Responsibilities

Own and extend LLM and agent evaluation pipelines; develop representative task suites; measure behavior at the trajectory level; research and prototype adaptive model-routing strategies; take promising methods from offline evaluation through traffic replay, shadow testing, and internal pilots.

Seniority

Senior, hands-on IC