Data Science Expert
Core
Define evaluation frameworks and assess AI-generated outputs for real-world data science tasks including analyses, models, and dashboards.
Role type
Senior IC data science evaluation expert (AI quality assurance)
Builds
Evaluation frameworks and structured feedback for AI systems
Domain
Artificial Intelligence / Data Science
Deliverable
production ML models
Required skills
Experimental design, A/B testing, Statistical analysis, SQL, Python, Metric development, Business recommendations
Preferred skills
LLMs, AI-assisted analytics, Machine learning workflows, Mentoring analytical teams
Responsibilities
Design task-specific evaluation criteria for data analyses, predictive models, dashboards, and experiment results; Evaluate AI-generated and human-produced work using objective scoring frameworks; Provide detailed written justifications for evaluations; Incorporate structured reviewer feedback to refine evaluation methodologies; Collaborate with cross-functional teams to improve AI training quality.
Seniority
Senior, hands-on IC