AI Data Scientist
Core
Designing metrics frameworks and analyzing AI model performance to guide product decisions for a generative AI platform.
Role type
Senior IC data scientist (generative AI)
Builds
Metrics frameworks, evaluation systems, and statistical models for AI-generated content quality
Domain
Generative AI / SaaS
Deliverable
production ML models
Required skills
Statistical modeling, A/B test design, large-scale data analysis, metrics framework construction, LLM/generative AI evaluation, SQL, Python
Preferred skills
Experience mentoring data scientists, agentic coding workflows
Technologies
dbt, Snowflake
Responsibilities
Build frameworks to understand AI model performance and user behavior; Dig into AI model outputs to identify quality gaps and create evals; Partner with engineering to define quality metrics for AI-generated content; Develop statistical models to empower product teams; Design and analyze large-scale A/B tests
Seniority
Senior, hands-on IC with mentorship