Member of Technical Staff - Pre-Training Evals Infra
Core
Build evaluation infrastructure for pre-training models to assess capabilities, guide training decisions, and ensure readiness for release.
Role type
Senior IC infrastructure engineer (ML evaluation systems)
Builds
Distributed evaluation execution systems, APIs, data models, and researcher-facing tools for model assessment
Domain
Artificial Intelligence / Machine Learning Infrastructure
Deliverable
production ML models
Required skills
Python, C++, Rust, distributed systems design, concurrency, memory management, system performance optimization, API design, product judgment
Preferred skills
Model inference knowledge, debugging complex system issues, translating research needs into tools
Technologies
Python, C++, Rust
Responsibilities
Design evaluation abstractions and APIs; build scalable distributed execution infrastructure; optimize inference and evaluation performance; establish correctness and reproducibility via testing; build researcher-facing dashboards; monitor evaluation workloads and triage failures
Seniority
Senior, hands-on IC
