Staff Applied AI Scientist
Core
Build and maintain continuous production evaluation frameworks for an AI Coach system, ensuring robust performance, safety, and quality at scale.
Role type
Staff Applied AI Scientist (LLMOps & Agentic Systems)
Builds
Observability and evaluation tooling for production AI agents, including LLM-powered analysis tools and guardrails.
Domain
Employee Experience Platform / AI Coaching / LLMOps
Deliverable
production ML models | product features
Required skills
Production agentic systems design, Context engineering (RAG, memory, compression), LLM evaluation (LLM-as-judge, human-in-the-loop), Observability tooling (Langfuse, traces), Longitudinal measurement, Model selection and routing, Safety guardrails design, Technical writing
Preferred skills
Postgraduate degree in ML/CS/Applied Maths, Public writing or talks in eval/LLMOps, Experience scaling eval practices across teams
Technologies
Langfuse, Claude Code, Cursor, Codex
Responsibilities
Own the end-to-end feedback loop for prompt engineering and continuous improvement; Design and run longitudinal evaluations with alerting; Contribute to agentic orchestration architecture; Create and monitor safety guardrails for sensitive data; Enable other teams with reusable evaluation frameworks.
Seniority
Staff, hands-on IC with enablement focus