Associate Director, Clinical AI Evaluation and Responsible Deployment
Core
Create evidence systems to determine if clinical AI is useful, reliable, safe, and ready to scale for clinical development.
Role type
Associate Director, Clinical AI Evaluation and Responsible Deployment
Builds
Evaluation harnesses, curated test sets, simulation environments, automated regression suites, and analysis pipelines
Domain
Healthcare / Clinical Development / AI Evaluation
Deliverable
production ML models
Required skills
Python, SQL, experimental design, statistical inference, LLM evaluation, agentic systems evaluation, GCP/GxP compliance, failure analysis, red-teaming, traceability, mentorship
Preferred skills
Clinical trial conduct, medical monitoring, causal inference, weak supervision, MLOps/LLMOps
Technologies
Python, SQL
Responsibilities
Define evaluation strategy for clinical AI models; Build evaluation harnesses and automated pipelines; Translate expert judgment into task definitions and scoring rubrics; Evaluate complete workflows including retrieval and orchestration; Design approaches for noisy or delayed ground truth; Lead failure analysis and red-teaming; Establish traceability from intended use to release decisions; Align evaluation evidence with Quality and Regulatory expectations; Advise on progression from prototype to deployment; Create reusable evaluation components; Communicate findings and risks to technical and executive audiences; Mentor scientists and engineers in rigorous experimentation.