AI Quality Engineering Team Lead
Core
Establish trustworthy, scalable quality engineering for GenAI and agentic AI solutions by combining hands-on evaluation with team leadership to ensure AI solutions are accurate, safe, and reliable.
Role type
Senior IC AI Quality Engineering Team Lead
Builds
Reusable quality practices, automated test pipelines, evaluation components, and auditable findings for multiple AI products.
Domain
Artificial Intelligence / Generative AI / Quality Engineering
Deliverable
production ML models
Required skills
AI evaluation, LLM/RAG/agent-based application testing, Python, API/log/trace analysis, SQL, automated testing, CI/CD, risk assessment, dataset management, stakeholder influence, team coaching
Preferred skills
Project leadership, AI observability tools (DeepEval, Arize), shared framework design, responsible AI/model risk knowledge, regulated industry experience, cloud AI systems exposure
Technologies
Python, SQL, DeepEval, Arize, CI/CD pipelines
Responsibilities
Lead quality engineering approach from risk assessment to release; Evaluate multi-turn, RAG, and agentic AI behavior; Build representative datasets and assess quality; Assess correctness, hallucination, safety, and bias; Investigate unexpected outcomes across data and orchestration; Build reusable evaluation components and automated pipelines; Integrate evaluation into CI/CD and observability workflows; Coach and guide AI Quality Engineers; Partner with product, engineering, and risk stakeholders.
Seniority
Senior, hands-on IC with leadership responsibilities
