Software Engineer, Evaluation Platform / Infra
Core
Design and build a self-serve platform for authoring, running, tracking, and analyzing model evaluations to guide frontier AI research and model releases.
Role type
Senior IC software engineer (evaluation platform & infrastructure)
Builds
A scalable, self-serve evaluation platform supporting standardized benchmarks and internal experiments for frontier AI models.
Domain
Artificial Intelligence / Machine Learning Infrastructure
Deliverable
production ML models
Required skills
Python, Rust, React, TypeScript, distributed systems, data pipelines, database design, API development, system reliability, cross-stack ownership
Preferred skills
Framework/SDK development, workflow orchestration, sandboxed environments, large-scale data processing, model-based grading, human evaluation, synthetic data generation, startup experience
Technologies
Python, Rust, React, TypeScript
Responsibilities
Design and maintain the evaluation platform end-to-end; build flexible abstractions for tasks, environments, and graders; ensure result reproducibility via versioning and observability; partner with researchers to convert bespoke workflows into self-serve systems; collaborate with Research Tooling and cross-functional teams.
Seniority
Senior, hands-on IC