Staff+ Software Engineer, Safeguards Evals
Core
Design experiments, build datasets, and ship infrastructure to evaluate AI model safety, agentic systems, and misuse detection.
Role type
Staff+ Software Engineer (AI Safety & Evaluation)
Builds
Evaluation harnesses, test datasets, and pipelines for model training and agent changes.
Domain
AI Safety, Trust & Safety, Large Language Models (LLMs), Agentic Systems
Deliverable
production ML models
Required skills
LLM evaluation frameworks, data pipeline engineering, experimental design, data analysis, agentic system reasoning, synthetic data generation, distributed systems
Preferred skills
Red teaming, adversarial testing, jailbreak research, content moderation, prompt engineering, RL environments
Technologies
LLMs, agentic systems, data pipelines, RL environments
Responsibilities
Design experiments to improve evaluation quality and validate grading accuracy; Build and own evaluation harnesses for agentic investigation systems; Construct high-quality eval datasets representing real-world misuse; Ship research into evaluation and release pipelines.
Seniority
Staff+, hands-on IC with strategy & mentorship
