Senior Data Scientist - M365 Copilot, Agents & Platform - Turing Team
Core
Design and build evaluation datasets and metrics for Microsoft 365 Copilot and AI agents to assess response quality across enterprise and consumer scenarios.
Role type
Senior Data Scientist (AI Evaluation & Product Strategy)
Builds
Representative evaluation datasets, decision-relevant metrics, and dashboards for Microsoft 365 Copilot and AI agents.
Domain
Enterprise AI / Large Language Models / Data Evaluation
Deliverable
production ML models
Required skills
Python, SQL, KQL, statistical analysis, study design, rubric creation, cross-functional collaboration, data visualization, product strategy, mentorship
Preferred skills
Domain expertise in AI/ML evaluation, experience with multimodal data, competitive analysis
Technologies
Python, SQL, KQL, R
Responsibilities
Prepare and analyze large-scale datasets (text, logs, feedback); design evaluation coverage for text/voice inputs and conversations; partner with engineering and product teams on measurement strategies; conduct side-by-side evaluations and assess statistical significance; develop metrics and dashboards to inform product decisions; mentor team members on analytical code and study design.
Seniority
Senior, hands-on IC with mentorship responsibilities