Staff Technical Product Manager – AI/LLM expertise + AI Evaluation Science
Core
Establish and scale AI Evaluation as a core engineering capability to make trusted Evals the evidence layer for AI across the DT organization.
Role type
Staff Technical Product Manager (AI Evaluation Science & Platform Strategy)
Builds
Enterprise-specific AI evaluation strategies, custom metrics, rubrics, datasets, and Eval-Driven Development practices for AI agents and workflows.
Domain
Enterprise AI / LLM Operations / Quality Engineering
Deliverable
production ML models | product features
Required skills
AI/LLM expertise, Evaluation Science, Quality Engineering, Platform Strategy, LLM-as-a-Judge, rubric design, agent evaluation, AI observability, CI/CD integration
Preferred skills
Servicenow AI Platform experience
Responsibilities
Partner with platform teams to drive adoption of out-of-the-box Evals; define custom evaluation strategies for enterprise AI applications; embed Evals into the full AI lifecycle from definition to production monitoring; establish evaluation coverage and quality thresholds for AI teams.
Seniority
Staff, strategic IC with cross-functional influence
