AI Agent Evaluator, Remote- Contract
Freelance / independent contractor
Core
Evaluate AI agents by running real-life errands (bills, bookings, shopping) and scoring their performance from a user's perspective.
Role type
Contractor AI agent evaluator
Builds
User experience data and quality scores for AI agents
Domain
Artificial Intelligence / Consumer Technology
Deliverable
production ML models
Required skills
ChatGPT, Muse, AI agents, Attention to Detail, Following Instructions, Screen Recording
Preferred skills
Engineering, Business Management, Office/Operations experience
Technologies
ChatGPT, Muse
Responsibilities
Run real-life scenarios with AI agents, approve or stop agents at key steps, screen record every attempt, score runs on completion and quality, share written feedback
Seniority
Mid-level (2-3 years experience)