Senior Product Manager, Observability Tools
Core
Own the data layer and quality tools for ASAPP's GenerativeAgent platform, enabling builders and customers to simulate, test, review, and evaluate AI agent behavior before and after production.
Role type
Senior Product Manager (AI Observability & Quality Tools)
Builds
Simulation agents, evaluation surfaces, mistake monitoring, structured data evals, Previewer, Conversation Explorer
Domain
Generative AI / Conversational AI / Customer Experience (CX)
Deliverable
product features
Required skills
Product management in enterprise B2B SaaS, LLM/AI agent product design, simulation and testing tooling, review/audit tooling for unstructured content, discovery and requirements translation, stakeholder influence
Preferred skills
Contact center or CX analytics experience, evaluation frameworks for generative AI, startup environment experience
Technologies
GenerativeAgent, simulation agents, reasoning traces, tool-use logs
Responsibilities
Lead product for scenario creation tools to validate configuration changes; Define mistake monitoring and structured data evaluation surfaces; Own the Previewer tool for simulating configurations; Lead the Conversation Explorer for auditing production interactions; Partner with Sales and Customer Success to demonstrate simulation-driven quality; Collaborate with Research and Engineering on agent architecture and evaluation methodology; Manage trust and UX for the review experience
Seniority
Senior, hands-on IC
