CareerPlan

Software Engineer, Evaluation Platform / Infra

San FranciscoFull-time$300k–$350k2026-08-31 → 2026-11-19
On-site · United StatesVisa sponsorship offered

Core

Design and build a self-serve platform for authoring, running, tracking, and analyzing model evaluations to guide frontier AI research and model releases.

Role type

Senior IC software engineer (evaluation platform & infrastructure)

Builds

A scalable, self-serve evaluation platform supporting standardized benchmarks and internal experiments for frontier AI models.

Domain

Artificial Intelligence / Machine Learning Infrastructure

Deliverable

production ML models

Required skills

Python, Rust, React, TypeScript, distributed systems, data pipelines, database design, API development, system reliability, cross-stack ownership

Preferred skills

Framework/SDK development, workflow orchestration, sandboxed environments, large-scale data processing, model-based grading, human evaluation, synthetic data generation, startup experience

Technologies

Python, Rust, React, TypeScript

Responsibilities

Design and maintain the evaluation platform end-to-end; build flexible abstractions for tasks, environments, and graders; ensure result reproducibility via versioning and observability; partner with researchers to convert bespoke workflows into self-serve systems; collaborate with Research Tooling and cross-functional teams.

Seniority

Senior, hands-on IC