SWE-Bench AI Task Auditor - Freelance AI Trainer Project
Core
Evaluate software engineering tasks for technical accuracy, realism, and reproducibility to improve AI training workflows.
Role type
Freelance AI Task Auditor (Software Engineering)
Builds
High-quality, reliable AI training datasets and evaluation benchmarks
Domain
Artificial Intelligence / Software Engineering
Deliverable
production ML models
Required skills
Software engineering expertise, codebase navigation, test failure diagnosis, logic error identification, technical validation, problem-solving, independent project management
Preferred skills
Familiarity with SWE-Bench-style tasks and evaluation workflows
Technologies
None specified
Responsibilities
Evaluate software engineering tasks for technical accuracy and alignment with standards; Review task codebases, integrations, and tests to identify weaknesses; Test and troubleshoot complex technical scenarios; Investigate codebase integration problems and logic errors; Provide actionable feedback to task creators; Apply professional software engineering judgment to assess realistic development scenarios
Seniority
Senior, hands-on IC