Principal QE Engineer/Architect - Automation Testing (10+ Years)- 11936
Core
Architect and develop custom AI testing frameworks, evaluation pipelines, and proprietary AI tools to test, evaluate, and validate AI-driven products and non-deterministic models.
Role type
Principal AI Quality Engineering Architect
Builds
Custom AI test automation frameworks, synthetic data generators, LLM evaluation harnesses, prompt regression suites, and self-healing test engines.
Domain
Artificial Intelligence / Machine Learning / Quality Engineering
Deliverable
production ML models
Required skills
AI framework architecture, LLM evaluation, agentic frameworks (LangChain, AutoGen), data quality standards, non-deterministic model testing, technical architectural leadership, cross-functional team leadership, strategic problem solving.
Preferred skills
Experience building side projects, rapid prototyping of pluggable AI components, industry representation.
Technologies
LangChain, AutoGen, MCP, LLMs, AI testing tools.
Responsibilities
Design and build custom test automation frameworks for AI-native applications; develop proprietary AI-driven tooling for model evaluation; provide executive technical advice on AI safety and quality gates; lead strategic initiatives for enterprise quality engineering; mentor senior engineers on AI tool adoption.
Seniority
Principal, strategic architect & technical leader
