Physician (MD/DO) - Medical AI Evaluation
Core
Evaluate clinical EHR vignettes and score clinical reasoning quality for medical AI systems.
Role type
Independent contractor medical AI evaluator
Builds
AI evaluation datasets and benchmark items
Domain
Healthcare + Artificial Intelligence
Deliverable
production ML models
Required skills
US medical license (MD/DO), clinical reasoning, differential diagnosis, evidence-based guidelines application, written English (C1+)
Preferred skills
Board certification, recent direct patient care, prior medical AI evaluation or annotation experience
Technologies
Clinical EHR systems, AI evaluation platforms
Responsibilities
Review written clinical cases across multiple medical systems, score vignette accuracy and answer correctness, write short rationales for reasoning chains, participate in blind reads and physician adjudication