Director AI Evaluation
Core
Defines, proves, and sustains quality standards across an enterprise AI portfolio (internally built and vendor-provided) to ensure clinical safety and efficacy for patients and clinicians.
Role type
Director of AI Evaluation (Hands-on technical leader & people manager)
Builds
Evaluation methodologies, validation studies, production monitoring plans, and reusable playbooks for AI systems.
Domain
Healthcare / Clinical AI / Machine Learning Evaluation
Deliverable
production ML models | dashboards & analysis
Required skills
People leadership (managing technical staff), experimental design, causal inference, model evaluation in production, LLM/generative AI evaluation, Python, SQL, fairness and equity evaluation, technical writing.
Preferred skills
Healthcare or clinical industry experience, regulated-industry experience.
Responsibilities
Set quality standards for all AI initiatives, manage data science and evaluation teams, design validation studies and monitoring plans, translate failure modes into measurable metrics, track system performance and user adoption, connect AI systems to clinical outcomes, monitor equity across subgroups.
Seniority
Director, strategic leadership with hands-on technical execution
