Content Quality and Evaluation Specialist - AI Data Service Operations
Core
Define quality standards and judgment rules for AI training and evaluation data to ensure data trustworthiness and consistency for global content platforms.
Role type
Senior IC content quality and evaluation specialist (AI data operations)
Builds
Multilingual, multimodal data assets for LLM training and evaluation pipelines
Domain
Generative AI / Large Language Models / Content Governance
Deliverable
production ML models
Required skills
Data quality standards definition, judgment criteria unification, complex edge case analysis, quality metrics tracking, cross-functional alignment, policy interpretation, structured problem decomposition, data tool proficiency, LLM product familiarity
Preferred skills
Content policy development, operational guideline creation, machine-executable logic familiarity, labeling framework experience, dataset validation knowledge
Technologies
Excel, mainstream LLM products (ChatGPT, Claude, Gemini)
Responsibilities
Design and iterate data quality standards and judgment rules for LLM scenarios; Review and adjudicate data outputs against standards to ensure accuracy; Analyze complex edge cases to create reusable judgment rules; Track quality metrics and drive process improvements; Partner with production, product, algorithm, and policy teams to align on quality standards
Seniority
Senior, hands-on IC
