Principal Product Manager - Copilot Extensibility DevX
Core
Own product vision, strategy, and roadmap for Copilot plugin evaluation and quality measurement, establishing Responsible AI models and connecting findings with production telemetry to guide investments.
Role type
Principal Product Manager (AI/LLM Evaluation & Extensibility)
Builds
Copilot plugin evaluation frameworks, quality measurement systems, and developer extensibility workflows
Domain
AI/LLM, Developer Platforms, Extensibility
Deliverable
production ML models
Required skills
Product strategy, Responsible AI modeling, LLM evaluation methods, Data-driven prioritization, Cross-organizational leadership, Technical prototyping, API/SDK definition, Stakeholder influence
Preferred skills
Experience with nondeterministic outputs, Multi-step agent flows, LLM-as-judge scoring, Offline-to-online measurement
Technologies
CLI, IDE, CI/CD, Telemetry systems, APIs, SDKs
Responsibilities
Define evaluation methods for nondeterministic outputs and multi-step agent flows, Validate product concepts through AI-assisted prototypes, Represent strategy and lead cross-organizational decisions, Connect evaluation findings with production telemetry and business-value signals, Establish quality and Responsible AI models and release criteria, Build evaluation into CLI and IDE workflows
Seniority
Principal, strategy & mentorship
