Principal Applied Science Manager - Outlook Science Team
Core
Lead and grow a team of applied scientists focused on AI quality, evaluation, post-training, and product improvement for AI-powered communication, summarization, search, and agentic experiences.
Role type
Principal Applied Science Manager (AI Quality & Evaluation)
Builds
Scalable evaluation systems, post-training strategies, and production AI improvements
Domain
Generative AI, Large Language Models (LLMs), Agentic Systems, Information Retrieval
Deliverable
production ML models | product features
Required skills
Team leadership, AI quality evaluation, post-training strategies, experimentation design, telemetry analysis, stakeholder communication, scientific rigor, product judgment, cross-functional partnership
Preferred skills
PhD in CS/AI/ML/NLP, experience with agentic systems, deep LLM expertise, track record in improving production AI quality, publication or patent record
Technologies
LLMs, agentic systems, human-in-the-loop evaluation, LLM graders, shadow A/B testing, telemetry systems
Responsibilities
Define quality metrics for AI communication and search, build scalable evaluation systems, diagnose production failures, translate user feedback into product insights, develop post-training strategies, mentor senior scientists, partner with PM and engineering leaders
Seniority
Principal, hands-on IC with team leadership
