Senior Product Operations Manager, Evaluation Quality
Core
Own the quality bar for Harvey's human evaluations to make outputs decision-grade for Product, Engineering, and AI Research.
Role type
Senior Product Operations Manager (Evaluation Quality)
Builds
Human Data signal, evaluation guidelines, operational dashboards
Domain
Legal Tech / AI Evaluation
Deliverable
production ML models
Required skills
quality program management, rubric design, inter-rater reliability, vendor management, cross-functional coordination, documentation, data interpretation
Preferred skills
legal tech experience, multi-market evaluation, LLM-as-judge familiarity
Technologies
AI tools for data interpretation
Responsibilities
Define evaluation guidelines and gold references, standardize evaluation methodology, manage contract-attorney onboarding and calibration, run QA on vendor deliverables, build operational dashboards for rater capability, partner with Applied Legal Researchers on evaluation alignment
Seniority
Senior, hands-on IC