Product Manager II, AI Evaluations
Core
Own AI Evaluations to assess, launch, and manage frontier AI models powering Box Agents and enterprise products.
Role type
Senior IC Product Manager (AI Evaluations)
Builds
AI model evaluation pipelines, datasets, and production-ready model selections for Box Agents.
Domain
Enterprise AI / Content Management
Deliverable
production ML models
Required skills
Product management, statistical literacy, Python, SQL, notebook usage, cross-functional leadership, external partnership management, technical communication, ambiguity resolution, model lifecycle management.
Preferred skills
AI/ML product experience, model benchmarking, vendor management.
Technologies
Python, SQL, notebooks, OpenAI, Anthropic, Google/Gemini.
Responsibilities
Run rigorous evaluations across frontier models; define high-quality evaluation datasets and graders; drive synthetic data generation pipelines; own online evaluations from production usage; lead model launches end-to-end; manage external narrative and PR for model launches; build relationships with model providers; decide model production readiness; assess compliance and pricing for models.
Seniority
Senior, hands-on IC
