Contract Student Worker – AI Validation and Benchmarking Engineer (40hrs / week Full-Time)
Core
Supporting the end-to-end validation pipeline for AI tools by maintaining test datasets, running benchmarks, and measuring agent accuracy across routing decisions, classification labels, and structured output fields.
Role type
AI validation and benchmarking engineer (student)
Builds
Evaluation infrastructure and ground truth datasets for autonomous vehicle AI agents
Domain
Autonomous vehicles / AI validation
Deliverable
production ML models
Required skills
Python, evaluation metrics (precision, recall, F1, confusion matrices), structured data handling (CSV, JSON), reproducible analysis scripting
Preferred skills
LLM-based systems, prompt engineering, AI agent evaluation, Jira, Slack
Technologies
Python, Cursor, Claude
Responsibilities
Run and maintain the benchmark pipeline, analyze results to identify routing errors and regressions; Build and expand ground truth datasets; Identify gaps in benchmark validation and support building evaluation infrastructure; Develop new evaluation dimensions; Investigate failure modes in agent outputs; Write scripts to automate data collection, result parsing, and metric reporting; Document findings and present results to the team
