AI Policy Generalist - Seattle Onsite
Core
Evaluate user requests and AI model responses against complex customer policies to determine the most defensible classification, distinguishing between closely related labels and edge cases.
Role type
Senior IC AI Policy Generalist (Evaluation)
Builds
Consistent, well-reasoned evaluations of AI model behavior for frontier AI labs and enterprise customers
Domain
AI Safety & Policy Evaluation
Deliverable
production ML models
Required skills
Policy interpretation, nuanced case distinction, evidence-based rationale writing, calibration participation, rapid learning of new taxonomies, intellectual curiosity, clear written communication
Preferred skills
Prior AI evaluation/RLHF experience, familiarity with LLM outputs (ChatGPT, Claude, Gemini), experience with regulatory language or editorial standards, comfort with sensitive content
Technologies
LLMs (ChatGPT, Claude, Gemini)
Responsibilities
Evaluate user requests and model responses within conversation context, distinguish between closely related labels and severity levels, write concise evidence-based rationales citing policy language, identify policy gaps and contradictions, participate in calibration discussions, update judgment based on new guidance
Seniority
Senior, hands-on IC