Tech Lead Data Scientist, AI Evaluation & Monitoring
Core
Principal technical expert guiding the evaluation, monitoring, and optimization of AI systems in production for a healthcare enterprise.
Role type
Senior IC Tech Lead Data Scientist (AI Evaluation & Monitoring)
Builds
Reusable evaluation playbooks, LLM-as-Judge frameworks, simulation harnesses, and production monitoring metrics.
Domain
Healthcare / AI Safety & Evaluation
Deliverable
production ML models
Required skills
Experimental design, causal inference, LLM/generative AI evaluation, fairness and equity analysis, Python, SQL, drift detection, technical mentorship
Preferred skills
Healthcare/clinical domain experience, MS or PhD in quantitative field
Technologies
Python, SQL, LLM-as-Judge, cloud-native data platforms
Responsibilities
Define technical evaluation methodology for enterprise AI programs, design validation studies and monitoring plans, mentor data analysts on statistical rigor, translate failure modes into measurable metrics, review analysis code and study designs.
Seniority
Senior, hands-on IC with technical leadership
