AI Agentic Workflow Reviewer
Core
Evaluate complex, real-world agentic AI workflows to ensure agents complete tasks safely, respect user intent/consent, and adhere to policies within isolated test environments.
Role type
Senior IC AI Agentic Workflow Reviewer
Builds
Evaluation frameworks and safety calibration for generative AI agents
Domain
Generative AI / AI Agents
Deliverable
production ML models
Required skills
Judgment-intensive review, rubric-based decision-making, privacy guardrails application, safety escalation, task trajectory analysis, user consent evaluation
Preferred skills
AI/ML concepts familiarity, ambiguous scenario navigation
Technologies
Isolated test environments
Responsibilities
Review multi-step agent task trajectories against rubric-based criteria; Evaluate agent respect for user agency and informed consent; Apply privacy guardrails for sensitive data; Verify environment resets to prevent cross-contamination; Flag and escalate safety-relevant or ambiguous findings
Seniority
Mid-level, hands-on IC