Member of Technical Staff - Applied AI Software Engineer, Health
Core
Design and build evaluation systems for LLMs in healthcare, run experiments on prompting techniques, and improve internal tooling for benchmarking and regression testing.
Role type
Applied AI Software Engineer (LLM Evaluation & Healthcare)
Builds
Evaluation systems, benchmarking frameworks, and regression testing capabilities for LLMs
Domain
Healthcare technology + Large Language Models (LLMs)
Deliverable
production ML models
Required skills
Python programming, machine learning research, LLM harness engineering, agent performance evaluation, data engineering for text datasets, cross-functional collaboration
Preferred skills
Experience in healthcare technology, 0 to 1 product experience, bias towards shipping
Technologies
Python, LLMs, internal benchmarking tools
Responsibilities
Translate research into benchmarks, co-author product roadmap, design evaluation systems for LLMs in healthcare, run experiments on prompting techniques, improve internal tooling for evaluations, design benchmarking and regression testing capabilities
Seniority
Individual Contributor (0 to 1 experience)
