CareerPlan

Member of Technical Staff - Evaluations

USAFull-time2026-11-11 → 2026-11-16
Hybrid

Core

Own the internal evaluation suite and infrastructure end-to-end, designing held-out evaluations for agentic workflows, computer use, and cybersecurity, and driving post-training decisions.

Role type

Senior IC machine-learning evaluation engineer

Builds

Reliable, optimized, reproducible, and scalable evaluation infrastructure and benchmarks

Domain

AI safety and model evaluation

Deliverable

production ML models

Required skills

evaluation design, contamination handling, reproducibility, software engineering, infrastructure debugging, benchmark creation, variance analysis, judge bias mitigation

Preferred skills

model training, research publications, deep systems expertise, small edge models

Technologies

evaluation frameworks, infrastructure tools

Responsibilities

Design held-out evaluations for capabilities including agentic workflows, computer use, and cybersecurity; Build evaluations targeted at small edge models and debug unexpected scores; Convert customer and open-source feedback into environments for data, training, and evaluation; Release open-source evaluations, write papers, and publish failure-mode reviews; Drive post-training decisions through internal evaluations

Seniority

Senior, hands-on IC