Head of Data Labeling
Core
Lead the Data Labeling team to build annotation guidelines, quality standards, and evaluation frameworks for AI safety and reliability.
Role type
Senior IC Head of Data Labeling (AI Safety)
Builds
Scalable data annotation workflows, evaluation frameworks, and reporting dashboards for LLMs and AI systems.
Domain
AI Safety, Machine Learning Operations, Generative AI
Deliverable
production ML models
Required skills
Team leadership, operational process design, AI model evaluation, vendor management, data quality assurance, project prioritization, metric monitoring, tooling improvement, stakeholder communication, distributed team management
Preferred skills
RLHF, preference data collection, safety evaluations, benchmark creation, prompt engineering, SQL, Python, internal platform building, linguistics, cognitive science
Technologies
LLMs, annotation tools, reporting dashboards
Responsibilities
Hire, coach, and manage the Data Labeling team; define annotation guidelines and quality standards; develop QA processes and auditing systems; partner with researchers to translate objectives into workflows; prioritize labeling projects; monitor operational metrics; improve annotation tooling and automation; lead AI evaluation projects; analyze disagreement patterns; manage vendor relationships; build reporting dashboards.
Seniority
Senior, hands-on IC with leadership responsibilities
