Senior Machine Learning Engineer - Orchestration - Applied AI & LLMs (x/f/m)
Core
Building robust evaluation pipelines and defining evaluation strategies for agentic AI systems to ensure safety, reliability, and quality of an AI Health Assistant for patients and practitioners.
Role type
Senior Machine Learning Engineer (LLM Evaluation & Orchestration)
Builds
Automated evaluation pipelines, metrics, protocols, and datasets for agentic AI systems
Domain
Healthcare / Large Language Models / Agentic AI
Deliverable
production ML models
Required skills
LLM evaluation, agentic system evaluation, experiment design, metric definition, evaluation automation, cross-functional collaboration, research-to-production bridging
Preferred skills
Clinical/medical domain knowledge, healthcare AI ethics/regulatory sensitivity
Technologies
Python, LLMs (GPT, Claude, Llama, BERT), agentic frameworks
Responsibilities
Define and own evaluation strategy (metrics, protocols, datasets, tooling); Implement and maintain automated evaluation pipelines; Run systematic experiments to assess reasoning, factuality, robustness, and user experience; Collaborate with model developers and research scientists; Contribute to research and knowledge sharing on LLM evaluation methodologies
Seniority
Senior, hands-on IC