Staff Applied AI Scientist
Core
Building observability and evaluation frameworks for a Coach AI system to ensure continuous production quality, real-time performance diagnosis, and sustainable scaling of agentic workflows.
Role type
Staff Applied AI Scientist (LLMOps & Agentic Systems)
Builds
Production-ready AI coaching agents with robust evaluation, monitoring, and safety guardrails for employee engagement platforms.
Domain
HR Tech / People Science / Generative AI
Deliverable
production ML models | product features
Required skills
Agentic system orchestration, RAG and context engineering, LLM evaluation (LLM-as-judge, human-in-the-loop), Observability tooling (Langfuse), Longitudinal measurement, Model selection and cost optimization, Guardrails and safety design, Technical writing and enablement
Preferred skills
Postgraduate degree in ML/CS/Applied Maths, Public writing or talks in eval/LLMOps, Experience scaling eval practices across teams
Technologies
Langfuse, Claude Code, Cursor, Codex, LLMs (various providers)
Responsibilities
Own the end-to-end feedback loop for prompt engineering and continuous improvement; Design and optimize context engineering for agentic flows; Design and run longitudinal evaluation systems; Contribute to agent architecture and drive structural changes based on eval findings; Make model selection and routing decisions; Create and monitor guardrails for sensitive people data; Enable other teams with reusable evaluation frameworks.
Seniority
Staff, hands-on IC with enablement focus