Research Engineer, Benchmarks
Core
Design, implement, and maintain internal benchmarks for evaluating frontier AI agents on domain-specific tasks.
Role type
Research Engineer, Benchmarks
Builds
Domain-specific evaluation benchmarks and infrastructure for frontier AI agents
Domain
Artificial Intelligence / Machine Learning / Agent Evaluation
Deliverable
production ML models
Required skills
Python, Docker, Linux, AI benchmark design, evaluation infrastructure, statistical analysis, technical writing
Preferred skills
Published research on AI benchmarks, reinforcement learning pipelines, data generation, agent evaluation experience
Responsibilities
Design and implement benchmarks for frontier agents, partner with SMEs to define evaluation criteria, build scalable execution infrastructure, develop metrics to assess benchmark reliability, validate results against real-world needs, write technical documentation
Seniority
Mid-level, hands-on IC