Senior Manager, AI Performance & Operations
Core
Own production performance and continuous improvement of core agentic workflows for healthcare service and experience, ensuring reliability, safety, and measurable outcomes for high-volume AI systems.
Role type
Senior Manager, AI Performance & Operations
Builds
Production-grade voice and text AI workflows, agentic orchestration systems, and evaluation frameworks for healthcare service delivery.
Domain
Healthcare technology, AI operations, agentic workflows
Deliverable
production ML models | product features | dashboards & analysis
Required skills
LLMs and RAG production experience, agentic orchestration, evaluation system design (rubrics/scoring), diagnostic acumen for model/prompt/orchestration failures, active experimentation for optimization, strategic communication, outcome ownership
Preferred skills
Healthcare industry background, regulated workflows experience, contact center operations, high-stakes service quality
Technologies
LLMs, RAG, voice AI, agentic orchestration, LLM-as-judge workflows, golden datasets
Responsibilities
Monitor weekly production indicators (containment, escalation, complaints) and oversee dashboards; lead incident response for AI regressions and drive post-mortems; design scalable evaluation systems with human review queues; guide new workflows during incubation to ensure proper instrumentation and safety; run structured optimization cycles for model swaps and prompt changes; translate technical changes into plain language for leadership
Seniority
Senior, hands-on IC with portfolio oversight
