Research Engineer, Benchmarks
Core
Design, implement, and maintain benchmarks for evaluating AI agents on domain-specific tasks.
Role type
Research Engineer (AI Benchmarks & Evaluation Infrastructure)
Builds
Domain-specific evaluation benchmarks and infrastructure for AI agents
Domain
Artificial Intelligence / Machine Learning / Agent Evaluation
Deliverable
production ML models
Required skills
Python, Docker, Linux, AI benchmark design, evaluation infrastructure, statistical analysis, technical writing
Preferred skills
Published research on benchmarking, reinforcement learning pipelines, data generation, agent evaluation
Technologies
Python, Docker, Linux
Responsibilities
Design and implement benchmarks for AI agents; Build infrastructure to run models/agents at scale; Develop metrics and analyses for benchmark reliability; Collaborate with SMEs to translate workflows into evaluation tasks; Write technical documentation on benchmark methods and results
Seniority
Mid-level, hands-on IC
