Research Engineer, Benchmarks
Core
Design, implement, and maintain benchmarks to evaluate frontier AI agents on domain-specific workflows and realistic tasks.
Role type
Research Engineer (AI Benchmarks & Evaluation)
Builds
Benchmarks, evaluation infrastructure, and metrics for AI agents
Domain
Artificial Intelligence / Machine Learning / Agent Evaluation
Deliverable
production ML models
Required skills
Python, Docker, Linux, benchmark design, evaluation infrastructure, statistical analysis, metrics development, technical writing
Preferred skills
Reinforcement learning pipelines, data generation, agent evaluation, published papers on benchmarking
Technologies
Python, Docker, Linux
Responsibilities
Design and implement benchmarks for frontier AI agents; collaborate with subject-matter experts to translate workflows into tasks; build infrastructure for running models at scale; develop metrics and analyses for benchmark reliability; validate correlation with real-world evaluations; write documentation and reports
Seniority
Mid-level, hands-on IC