CareerPlan

Senior Backend Engineer, Inference Platform

United States - RemoteRemoteFull-time$200k–$250k2026-11-02 → 2026-11-19
Remote · from United States

Core

Own and scale the inference gateway that routes every model call, managing provider selection, fallback, caching, and cost metering for AI search infrastructure.

Role type

Senior Backend Engineer (Inference Platform)

Builds

Inference gateway, control plane, routing logic, and cost attribution systems for AI model serving.

Domain

AI Infrastructure / Large Language Model Serving

Deliverable

production ML models

Required skills

Model serving (vLLM, SGLang), continuous batching, KV/prefix caching, proxy/gateway architecture, streaming responses, Python, Kubernetes, cost metering, request-path enforcement.

Preferred skills

Experience at inference provider or GPU cloud, open-source serving stack contributions, fine-tuning/evaluating models, usage metering/billing at scale.

Responsibilities

Own the inference gateway including provider/model selection and fallback; build intelligence layer for autonomous model selection; make routing decisions accounting for cache behavior and latency; build cost attribution and metering; build request-path enforcement; ship changes in shadow mode.

Seniority

Senior, hands-on IC