Member of Technical Staff - Evaluations
Core
Own the internal evaluation suite and infrastructure end-to-end, designing held-out evaluations for agentic workflows, computer use, and cybersecurity, and driving post-training decisions.
Role type
Senior IC machine-learning evaluation engineer
Builds
Reliable, optimized, reproducible, and scalable evaluation infrastructure and benchmarks
Domain
AI safety and model evaluation
Deliverable
production ML models
Required skills
evaluation design, contamination handling, reproducibility, software engineering, infrastructure debugging, benchmark creation, variance analysis, judge bias mitigation
Preferred skills
model training, research publications, deep systems expertise, small edge models
Technologies
evaluation frameworks, infrastructure tools
Responsibilities
Design held-out evaluations for capabilities including agentic workflows, computer use, and cybersecurity; Build evaluations targeted at small edge models and debug unexpected scores; Convert customer and open-source feedback into environments for data, training, and evaluation; Release open-source evaluations, write papers, and publish failure-mode reviews; Drive post-training decisions through internal evaluations
Seniority
Senior, hands-on IC