CareerPlan

AI Data Quality Analyst

Israel, Tel AvivFull-time2026-10-13 → 2026-11-22
Hybrid

Core

Evaluate LLM-generated conversations and text outputs, perform data labeling, and monitor production data to ensure quality and identify trends for AI model improvement.

Role type

AI Data Quality Analyst (LLM evaluation & labeling)

Builds

Reliable datasets for model training, benchmarking, and evaluation for Workday/Paradox AI products

Domain

Generative AI / Large Language Models (LLM)

Deliverable

production ML models

Required skills

Qualitative research, pattern detection in textual data, data annotation, LLM evaluation, cross-functional communication, English fluency

Preferred skills

Data labeling experience, LLM analysis experience, AI tool usage (Claude Code, Cursor)

Technologies

Web-based tools, labeling platforms, dashboards

Responsibilities

Review and evaluate LLM-generated conversations against quality criteria, perform high-quality annotation for model training, monitor real production data for quality trends, identify qualitative patterns and failure modes, analyze evaluation results and communicate insights to stakeholders, partner with AI Engineers and Product Managers on data needs

Seniority

Individual Contributor