CareerPlan

Expert Talent Pool, Feedback and Alignment (Senior, 5+ Years)

RemoteFull-time2026-11-15 → 2026-11-16

Core

Senior rater and reviewer evaluating challenging AI model outputs, calibrating other graders, and supporting the development of reliable AI systems.

Role type

Senior IC AI model evaluation specialist

Builds

Reliable, high-quality AI systems

Domain

Artificial Intelligence

Deliverable

production ML models

Required skills

AI model evaluation, rubric grading, fact-checking, policy assessment, rater calibration, domain expertise (math, code, law, medicine, finance), search relevance assessment

Preferred skills

AI data annotation, preference ranking, process labeling, search-quality rating

Responsibilities

Rank and rate expert-level model answers using detailed rubrics and written rationales; Evaluate model reasoning step by step in specialized fields; Fact-check detailed claims against primary sources; Assess answers against content policies, style guides, and edge cases; Review other raters' work, calibrate standards, and develop or refine grading guidelines; Assess search relevance and share feedback in live sessions

Seniority

Senior, hands-on IC