LLM - AI Quality Analyst (Personalization) - German
Core
Evaluate the quality of personalized AI responses in Google Gemini by designing prompts, assessing grounding and integration, and ranking model outputs.
Role type
AI Quality Analyst (Personalization)
Builds
Personalized AI interactions for Google Gemini
Domain
AI/ML, Personalization, Large Language Models
Deliverable
production ML models
Required skills
Creative prompt engineering, Analytical thinking, Attention to detail, Written communication, Data hygiene practices
Preferred skills
Data annotation experience, AI quality evaluation experience, Content moderation experience
Technologies
Google Gemini, Side-by-Side (SxS) evaluation interface
Responsibilities
Design multi-turn conversational prompts utilizing personal context, Evaluate model responses for grounding and integration quality, Stack-rank two model responses side-by-side, Write defensible rationales for comparisons, Extract and verify debug info from model outputs, Maintain data hygiene by deleting evaluation conversations
Seniority
Individual Contributor, Contractor