Expert Talent Pool, Feedback and Alignment (Senior, 5+ Years)
Core
Senior rater and reviewer evaluating challenging AI model outputs, calibrating other graders, and supporting the development of reliable AI systems.
Role type
Senior IC AI model evaluation specialist
Builds
Reliable, high-quality AI systems
Domain
Artificial Intelligence
Deliverable
production ML models
Required skills
AI model evaluation, rubric grading, fact-checking, policy assessment, rater calibration, domain expertise (math, code, law, medicine, finance), search relevance assessment
Preferred skills
AI data annotation, preference ranking, process labeling, search-quality rating
Responsibilities
Rank and rate expert-level model answers using detailed rubrics and written rationales; Evaluate model reasoning step by step in specialized fields; Fact-check detailed claims against primary sources; Assess answers against content policies, style guides, and edge cases; Review other raters' work, calibrate standards, and develop or refine grading guidelines; Assess search relevance and share feedback in live sessions
Seniority
Senior, hands-on IC