Senior QA Engineer (AI Solutions)
Core
Designing and executing quality assurance strategies for AI-enabled agent platforms and solutions, focusing on probabilistic testing for accuracy, grounding, and safety.
Role type
Senior SDET / QA Engineer (AI Systems)
Builds
Automated test suites, LLM evaluation harnesses, and regression frameworks for AI agents.
Domain
Artificial Intelligence / Agentic Workflows / RAG
Deliverable
production ML models | product features
Required skills
AI/LLM testing strategies, probabilistic evaluation, automation framework design, CI/CD integration, LLM-as-judge implementation, red-teaming, shift-left testing
Preferred skills
Multi-agent orchestration knowledge, RAG system testing, behavioral drift detection
Technologies
LLM evaluation frameworks, CI/CD pipelines, automated testing tools
Responsibilities
Define QA strategy for AI agents, build automated test suites across UI/API layers, design LLM evaluation harnesses, maintain golden datasets and regression suites, establish deployment gates for AI features
Seniority
Senior, hands-on IC