Senior Machine Learning Engineer - Orchestration - Applied AI & LLMs (x/f/m)
Core
Define evaluation strategies and maintain automated pipelines to ensure the safety, reliability, and quality of an AI Health Assistant (agentic AI system) serving millions of patients and practitioners.
Role type
Senior Machine Learning Engineer (LLM Evaluation & Orchestration)
Builds
Robust evaluation pipelines, metrics, and tooling for agentic AI systems
Domain
Healthcare / Large Language Models (LLMs) / Agentic AI
Deliverable
production ML models
Required skills
LLM evaluation, experiment design, metric definition, evaluation automation, agentic system assessment, cross-functional collaboration, research-to-production bridging
Preferred skills
Clinical/medical domain knowledge, sensitivity to ethical/regulatory challenges in healthcare AI
Technologies
GPT, Claude, Llama, BERT-like architectures, Python, TypeScript, Java, Rails, React Native
Responsibilities
Define evaluation strategy (metrics, protocols, datasets, tooling); Implement and maintain automated evaluation pipelines; Run systematic experiments on reasoning, factuality, and robustness; Collaborate with model developers and research scientists; Contribute to research on LLM evaluation methodologies
Seniority
Senior, hands-on IC
