Engineering Product Manager
Core
Lead the strategy, architecture, and organizational adoption of Cisco's end-to-end AI evaluation platform, including offline benchmarking suites, automated LLM judges, and continuous production evaluation systems.
Role type
Senior Engineering Product Manager (AI/ML Infrastructure)
Builds
Unified offline benchmarking suites, automated LLM judges, and continuous production evaluation systems for Cisco's global product portfolio.
Domain
Generative AI, Large Language Models (LLM), and AI infrastructure
Deliverable
production ML models
Required skills
AI/ML platform product management, model evaluation framework design, risk-based release criteria definition, multi-team platform adoption, automated regression detection, human-in-the-loop pipeline design
Preferred skills
Generative AI evaluation, RAG pipelines, autonomous multi-agent systems, AI evaluation toolkits (DeepEval, Ragas, LangSmith, TruLens), enterprise telemetry architectures, statistical evaluation methodologies, online evals for agent systems
Technologies
Amplitude, DeepEval, Ragas, LangSmith, TruLens
Responsibilities
Architect product vision and roadmap for AI evaluation platforms; Convert quality standards into objective test suites and review pipelines; Establish pre-deployment quality bars and release gating criteria; Connect offline evaluations with live telemetry and user feedback; Drive organization-wide adoption of standardized evaluation frameworks.
Seniority
Senior, hands-on IC