Director of Product Management, AI Evals
Core
Define the vision, strategy, and roadmap for Harness's AI Evaluation module, enabling engineering teams to measure, validate, and continuously improve AI and LLM-powered applications in production.
Role type
Director of Product Management (AI Evals)
Builds
AI-assisted experiences for eval authoring, failure analysis, and remediation recommendations within the software delivery lifecycle.
Domain
AI/ML evaluation, developer tooling, platform products, CI/CD pipelines
Deliverable
product features
Required skills
AI/ML evaluation methodologies, CI/CD pipeline design, AI observability and monitoring, agentic AI system evaluation, product strategy and P&L ownership, GTM partnership, customer engagement, competitive positioning, product metrics analysis
Preferred skills
Building AI/ML evaluation or observability platforms at scale, building products from the ground up, prompt engineering and fine-tuning pipelines, AI governance frameworks, cloud-native enterprise SaaS experience
Technologies
LLMs, CI/CD pipelines, agentic AI systems
Responsibilities
Design AI-assisted experiences for eval authoring, failure analysis, and remediation recommendations; Own the vision, strategy, roadmap, and P&L for the AI Evals module; Partner with engineering to deliver scalable solutions; Work with GTM teams on positioning and revenue growth; Engage directly with customers to shape the roadmap; Drive competitive positioning and track product metrics
Seniority
Director, hands-on IC with business ownership
