CareerPlan

Senior LLM Inference Performance & Evaluation Engineer

Singapore, SGFull-time2026-11-04 → 2026-11-16

Core

Build benchmark pipelines, launch gates, and automated regression detection for LLM inference performance, latency, and cost across model/runtime/provider options.

Role type

Senior IC LLM inference performance & evaluation engineer

Builds

Benchmark pipelines, model launch gates, automated regression detection systems, and routing/fallback recommendations

Domain

AI cloud / LLM inference infrastructure

Deliverable

production ML models

Required skills

LLM serving metrics (TTFT, TPOT/ITL, latency, throughput, concurrency, GPU utilization), Python, benchmark automation, statistical test design, CI/CD integration, Kubernetes

Preferred skills

Go, OpenAI/Anthropic APIs, vLLM/Dynamo/SGLang/Triton-style servers, SLO definition

Technologies

Python, Go, vLLM, Dynamo, SGLang, Triton, Kubernetes, OpenAI API, Anthropic API

Responsibilities

Build benchmark pipelines for TTFT, latency, throughput, and GPU utilization; Create model launch gates for compatibility and behavior; Maintain representative workloads using synthetic and replayed traffic; Compare model/runtime options and publish routing recommendations; Automate regression detection in CI/CD; Work with runtime engineers to identify bottlenecks and with SRE to convert results into SLOs

Seniority

Senior, hands-on IC