Contract Student Worker
Core
Run and maintain benchmark pipelines, analyze routing errors and regressions, and build ground-truth datasets to evaluate AI agent outputs.
Role type
Contract student worker (AI evaluation & validation)
Builds
Evaluation infrastructure, ground-truth datasets, and benchmark pipelines for AI agents
Domain
Artificial Intelligence / Machine Learning / Data Engineering
Deliverable
production ML models
Required skills
Python, data analysis, script automation, structured data handling (CSV/JSON), evaluation metrics (precision/recall/F1/confusion matrices), benchmark pipeline maintenance
Preferred skills
Familiarity with Cursor or Claude, experience with reproducible analysis scripts
Technologies
Python, CSV, JSON, Cursor, Claude
Responsibilities
Run and maintain benchmark pipelines, analyze results for routing errors and regressions, build and expand ground-truth datasets, write scripts to automate data collection and metric reporting, document findings and present results to the team
Seniority
Student / Contract
