Agentic AI Quality Engineer
Core
Design and execute quality assurance strategies for autonomous AI agents, focusing on safety, performance, and reliability in production environments.
Role type
Senior IC Agentic AI Quality Engineer
Builds
Production-ready agentic AI workflows, evaluation frameworks, and safety guardrails for enterprise clients.
Domain
Artificial Intelligence / Agentic Systems / MLOps
Deliverable
production ML models
Required skills
Agentic AI evaluation, Python, Prompt engineering, Adversarial testing, MLOps, Schema validation, LLM-as-a-Judge frameworks, CI/CD integration, Golden dataset curation
Preferred skills
TMAP or ISTQB Certification, Open-source contributions
Technologies
Python, MLflow, Galileo, PyRIT, Vector databases
Responsibilities
Automate evaluation of multi-turn conversations and long-horizon tasks; Implement safety filters and test for hallucinations and bias; Run red-team and prompt-injection tests; Embed test harnesses in CI/CD pipelines; Translate business risks into measurable quality criteria.
Seniority
Senior, hands-on IC
