Principal Applied Scientist
Core
Lead applied science strategy for AI-powered product areas, defining success metrics and operationalizing LLM evaluation frameworks across the product lifecycle.
Role type
Principal Applied Scientist (AI Strategy & Evaluation)
Builds
AI-powered product experiences, LLM evaluation frameworks, and experimentation pipelines
Domain
Artificial Intelligence / Large Language Models / Product Strategy
Deliverable
production ML models | product features | dashboards & analysis
Required skills
LLM evaluation, prompt engineering, RAG, agentic workflows, multimodal AI systems, offline experimentation design, MLOps, statistical analysis, product success metrics translation, technical leadership
Preferred skills
Responsible AI safety evaluations, red teaming, bias/fairness assessment, creating reusable evaluation platforms, influencing senior stakeholders
Technologies
LLMs, RAG, agents, telemetry, data pipelines
Responsibilities
Define success metrics for AI product experiences; Design and operationalize LLM evaluation frameworks; Partner with engineering and PM leadership to align science priorities; Mentor scientists and engineers; Drive fast iteration cycles through hypothesis formation and experimentation; Establish quality bars and decision-making frameworks for AI launches
Seniority
Principal, strategy & mentorship
